indexer

Indexer

This is a small Go indexer that fetches Ethereum event logs for a single contract/topic on Sepolia, enriches them with block headers, and persists compact records into a local bbolt database (indexer.db).

How to run it

Prereqs:

• Go 1.25.6 (see go.mod)
• Network access to Sepolia RPCs

Local:

• go run .
• go run . --print (prints up to 10 records after ingest)

Makefile shortcuts:

• make run
• make run-print
• make test

Docker:

• make docker-build
• make docker-run
• make docker-run-print

Notes:

• The database is created at ./indexer.db.
• The Docker run targets mount the repo as /data so indexer.db persists on your host.

Design / architecture notes

Core flow:

1. main.go builds a worker list from configured RPC endpoints.
2. indexer.New(...) resolves the target contract/topic and the block range.
3. The indexer splits the range into fixed-size chunks and feeds them to worker goroutines.
4. Each worker calls FetchLogs, sorts logs by block/log index, loads headers, and emits Record slices on a channel.
5. The store consumes ordered blocks and persists records to bbolt.

Concurrency:

• Workers run in goroutines and pull ranges from a channel.
• The store consumes a channel of record slices and commits in order.

Data model (fixed-size record):

Record (116 bytes)
┌────────────┬────────────┬──────────┬────────────────┬────────────────┬────────────────┐
│ block #    │ block time │ log idx  │ tx hash (32B)  │ parent hash    │ info root      │
│ uint64     │ uint64     │ uint32   │ [32]byte       │ [32]byte       │ [32]byte       │
└────────────┴────────────┴──────────┴────────────────┴────────────────┴────────────────┘

Key files:

• main.go: entrypoint + worker setup + CLI flag
• indexer/indexer.go: coordination, chunking, worker fan-out
• indexer/rpc.go: log fetch + retry, range selection
• store/store.go: bbolt persistence
• store/record.go: compact serialization

Tradeoffs / assumptions

• Hard-coded contract address, topic, and RPC endpoints in main.go.
• No reorg handling or confirmation depth; uses whatever the RPC returns.
• Simple retry policy (3 attempts, exponential backoff).
• Fixed chunk size (10k blocks) and record format (no schema versioning).
• Stores only logs from one topic and a single contract per run.

Improvement criteria (what good looks like)

• Take work from slow running/ non responsive RPC(s)
• Batch update on DB
• Resume from where it left [enabled by a flag]

Reliability:

• No data loss on transient RPC failures.
• Deterministic ordering of persisted records.
• Reorg-safe indexing with confirmation depth.

Performance:

• Sustains X blocks/sec or Y logs/sec on Sepolia without timeouts.
• Bounds memory usage while handling large ranges.

Operability:

• Configurable RPC endpoints, contract address, topic, and range.
• Clear progress metrics and retry diagnostics.
• Optional resume/checkpoint support to avoid re-indexing.

Correctness:

• Unit tests cover record serialization and store operations.
• End-to-end test validates a known block range against expected counts.