Documentation
¶
Index ¶
- Variables
- type ColumnCompressor
- func (cc *ColumnCompressor) AddItem(txn *ColumnTransaction) error
- func (cc *ColumnCompressor) Close() error
- func (cc *ColumnCompressor) CompressedBytes(column int) []byte
- func (cc *ColumnCompressor) NewTransaction() *ColumnTransaction
- func (cc *ColumnCompressor) Reset()
- func (cc *ColumnCompressor) UncompressedLen(column int) int
- type ColumnTransaction
- func (ct *ColumnTransaction) Float32(column int, v float32)
- func (ct *ColumnTransaction) Float64(column int, v float64)
- func (ct *ColumnTransaction) Int64(column int, v int64)
- func (ct *ColumnTransaction) Reset()
- func (ct *ColumnTransaction) Sint64(column int, v int64)
- func (ct *ColumnTransaction) Uint64(column int, v uint64)
- func (ct *ColumnTransaction) Write(column int, b []byte)
- type Compressor
- type JSONPayloadBuilder
- type OnErrItemTooBigPolicy
Constants ¶
This section is empty.
Variables ¶
var ( // ErrPayloadFull is returned when the payload buffer is full ErrPayloadFull = errors.New("reached maximum payload size") // ErrItemTooBig is returned when a item alone exceeds maximum payload size ErrItemTooBig = errors.New("item alone exceeds maximum payload size") )
Functions ¶
This section is empty.
Types ¶
type ColumnCompressor ¶ added in v0.74.0
type ColumnCompressor struct {
// contains filtered or unexported fields
}
ColumnCompressor builds columnar payloads while observing compressed and uncompressed size limits.
func NewColumnCompressor ¶ added in v0.74.0
func NewColumnCompressor(compression metricscompression.Component, numColumns int, maxCompressedSize, maxUncompressedSize int) ColumnCompressor
NewColumnCompressor creates a new instance
func (*ColumnCompressor) AddItem ¶ added in v0.74.0
func (cc *ColumnCompressor) AddItem(txn *ColumnTransaction) error
AddItem tries to add a transaction to the compressed payload.
Returns ErrPayloadFull if the transaction does not fit within current limits.
func (*ColumnCompressor) Close ¶ added in v0.74.0
func (cc *ColumnCompressor) Close() error
Close finishes compression of all pending data.
func (*ColumnCompressor) CompressedBytes ¶ added in v0.74.0
func (cc *ColumnCompressor) CompressedBytes(column int) []byte
CompressedBytes returns compressed bytes for a column.
func (*ColumnCompressor) NewTransaction ¶ added in v0.74.0
func (cc *ColumnCompressor) NewTransaction() *ColumnTransaction
NewTransaction creates a new transaction for the compressor.
func (*ColumnCompressor) Reset ¶ added in v0.74.0
func (cc *ColumnCompressor) Reset()
Reset clears compressor state and prepares it to build a new payload.
func (*ColumnCompressor) UncompressedLen ¶ added in v0.74.0
func (cc *ColumnCompressor) UncompressedLen(column int) int
UncompressedLen returns length of uncompressed data in a column.
type ColumnTransaction ¶ added in v0.74.0
type ColumnTransaction struct {
// contains filtered or unexported fields
}
ColumnTransaction contains data that is being added to the compressed stream
func (*ColumnTransaction) Float32 ¶ added in v0.74.0
func (ct *ColumnTransaction) Float32(column int, v float32)
Float32 adds a little-endian encoded float32 value to a column
func (*ColumnTransaction) Float64 ¶ added in v0.74.0
func (ct *ColumnTransaction) Float64(column int, v float64)
Float64 adds a little-endian encoded float64 value to a column
func (*ColumnTransaction) Int64 ¶ added in v0.74.0
func (ct *ColumnTransaction) Int64(column int, v int64)
Int64 adds a varint encoded integer to a column
func (*ColumnTransaction) Reset ¶ added in v0.74.0
func (ct *ColumnTransaction) Reset()
Reset prepares transaction to receive a new item
func (*ColumnTransaction) Sint64 ¶ added in v0.74.0
func (ct *ColumnTransaction) Sint64(column int, v int64)
Sint64 adds a zigzag encoded integer to a column
func (*ColumnTransaction) Uint64 ¶ added in v0.74.0
func (ct *ColumnTransaction) Uint64(column int, v uint64)
Uint64 adds a varint encoded unsigned integer to a column
func (*ColumnTransaction) Write ¶ added in v0.74.0
func (ct *ColumnTransaction) Write(column int, b []byte)
Write adds a sequence of bytes to a column
type Compressor ¶
type Compressor struct {
// contains filtered or unexported fields
}
Compressor is in charge of compressing items for a single payload
func NewCompressor ¶
func NewCompressor(input, output *bytes.Buffer, maxPayloadSize, maxUncompressedSize int, header, footer []byte, separator []byte, compressor compression.Compressor) (*Compressor, error)
NewCompressor returns a new instance of a Compressor
func (*Compressor) AddItem ¶
func (c *Compressor) AddItem(data []byte) error
AddItem will try to add the given item
func (*Compressor) Close ¶
func (c *Compressor) Close() ([]byte, error)
Close closes the Compressor, flushing any remaining uncompressed data
type JSONPayloadBuilder ¶
type JSONPayloadBuilder struct {
// contains filtered or unexported fields
}
JSONPayloadBuilder is used to build payloads. JSONPayloadBuilder allocates memory based on what was previously need to serialize payloads. Keep that in mind and use multiple JSONPayloadBuilders for different sources.
func NewJSONPayloadBuilder ¶
func NewJSONPayloadBuilder(shareAndLockBuffers bool, config config.Component, compressor compression.Component, logger log.Component) *JSONPayloadBuilder
NewJSONPayloadBuilder returns a new JSONPayloadBuilder
func (*JSONPayloadBuilder) BuildWithOnErrItemTooBigPolicy ¶
func (b *JSONPayloadBuilder) BuildWithOnErrItemTooBigPolicy( m marshaler.IterableStreamJSONMarshaler, policy OnErrItemTooBigPolicy) (transaction.BytesPayloads, error)
BuildWithOnErrItemTooBigPolicy serializes a metadata payload and sends it to the forwarder
type OnErrItemTooBigPolicy ¶
type OnErrItemTooBigPolicy int
OnErrItemTooBigPolicy defines the behavior when OnErrItemTooBig occurs.
const ( // DropItemOnErrItemTooBig skips the error and continues when ErrItemTooBig is encountered DropItemOnErrItemTooBig OnErrItemTooBigPolicy = iota // FailOnErrItemTooBig returns the error and stop when ErrItemTooBig is encountered FailOnErrItemTooBig )
Source Files