Blockchain intelligence for Web3: Building an in‑house analytics stack

Short summary

This guide gives you a straightforward, practical way to build your own blockchain intelligence stack, right in your own office. It's perfect for decision-makers who want to take control of their data and make informed choices. It covers a ton of ground, from selecting the right nodes to handling blob sidecar retention. Plus, it goes into using Solana Geyser streaming and dealing with the nitty-gritty of OP Stack fault-proofing. You’ll also come across some handy tips on MEV-aware telemetry, schemas, and dbt-driven modeling. It’s all loaded with practical configurations, APIs, and vendor options that you can dive into and start using this quarter!

---

Why build in‑house now (and what’s changed in 2024-2025)

So, Ethereum just launched its Dencun upgrade (EIP-4844), and it brought along some cool new features! One of the highlights is these "blob" transactions, which have a retention limit of about 4,096 epochs. To put that in simpler terms, that’s roughly 18 days. Pretty neat, right? This change has definitely made a big difference in cutting down L2 DA costs. So, if teams want to hold onto a complete history, they’ll have to take care of managing the blob sidecars on their own. So, each blob consists of 4,096 field elements, which adds up to 32 bytes. When you crunch the numbers, that gives you around 131,072 bytes, or roughly 128 KiB. Pretty neat, right? So, when it comes to blobs, we're aiming for a target of about 393,216 gas per block. The absolute max we can handle is around 786,432. This means that when we launch, we should expect to see roughly 3 target blobs and about 6 max blobs in each block.
If you’re interested, you can dive deeper into the details here.

When it comes to Ethereum finality, it’s still sitting at about 15 minutes. We're still working on getting single-slot finality all sorted out, and it's on our roadmap for the future! When you’re diving into your analytics and alerts, it’s super important to keep an eye on the difference between probabilistic states and finalized ones. Just a little heads-up to help you make sense of the data! Want more info? You can dive into the details here.

On June 10, 2024, Optimism took a big step forward by rolling out permissionless fault proofs on the OP Mainnet. Exciting times ahead! So, OP Stack rollups have hit what’s known as “Stage 1” decentralization! It’s a big step forward, but they’re still keeping a Security Council in place just in case. When it comes to figuring out capital latency, those windows for withdrawals really matter. They play a big role in how you set things up. If you're looking for more details, just check out this link. It's got all the info you need!

So, remember when Solana's state compression and cNFTs were just this neat little trick? Well, now they’ve totally evolved into something we see everywhere. It's pretty wild how fast things can change! We’ve really shaken things up in how we handle NFT and state data! Now, we’re using Merkle proofs and read APIs, moving away from just relying on token accounts. It’s a pretty exciting change! If you want to dive deeper into the topic, check it out here. There's a lot of great info waiting for you!

So, when we talk about PBS and MEV, the block-building world is pretty much dominated by a handful of specialized builders and relays, all thanks to mev-boost. So, what this really means is that "block provenance" is turning into a key thing to keep in mind when you're diving into your data warehouse analysis. If you want to dive deeper into the details, feel free to check out this resource. It's got all the info you need!

Here’s the deal: costs for DA have dropped, and things are moving faster than ever! To really get solid long-term insights, you need to pull together info from a bunch of different sources--like execution data, consensus/DA, and builders. And the key is to process all that info quickly!

---

Reference architecture (pragmatic and proven)

Think about tackling things in layers so you can handle staffing, testing, and budget monitoring more easily. Breaking it down like this can really help you stay organized and on top of everything!

1. The go-to source for truth (think of it as Nodes and DA).

If you're looking for an Ethereum execution layer (EL) client, you should definitely check out Erigon. It could be a great fit for you! This thing is really geared towards handling analytics traces efficiently, plus it’s quick and doesn’t take up much storage space--kind of like those parity-style trace APIs you might be familiar with. Another great choice is Reth. It's pretty modern and super fast, plus it meshes nicely with Geth and Parity-style tracing. If you're looking for something that can keep up, Reth could definitely be worth checking out! So, if you decide to use Erigon, just a heads-up: you'll need roughly 920 GB if you want to run it in full mode. And for the other option, it's about 1... By September 2025, we’re looking at 77 TB set aside for archive mode. If you're aiming for a minimal setup, you can expect to need about 350 GB. For the best performance, definitely go with NVMe storage! (docs.erigon.tech).

Oh, and make sure you keep the consensus client in mind! It's pretty essential for grabbing those blob sidecars and making sure everything's finalized. You just need to open up the /eth/v1/beacon/blob_sidecars endpoint and make sure to save the data it provides. So, here’s the deal with blob sidecars: they're shared all over the network, which means if you want to keep that historical data, you need to hang onto them for longer than about 18 days. Otherwise, you'll lose track of everything. (eips.ethereum.org).

• Lastly, if you're diving into Solana, you might want to think about setting up a validator with Geyser plugins. It could really enhance your experience! With this setup, you can easily separate indexing from the RPC load. This means you'll be able to stream account data, transaction statuses, and slot info straight to Kafka or PostgreSQL. It's a pretty smooth way to manage everything! (docs.solanalabs.com).

2) Ingestion and Streaming

• EVM Execution: Alright, let’s kick things off! First, you’ll want to start up the Erigon rpcdaemon, making sure to include the eth, debug, and trace modules. With this, you'll be able to use debug_traceTransaction and trace_filter. Hey, just a quick reminder to adjust the concurrency settings for those unexpected spikes in activity! It’ll really help keep things running smoothly. Check out these flags you might want to consider using. Of course, feel free to tweak them to fit your setup!

  rpcdaemon --http.api=eth,erigon,web3,net,debug,trace,txpool --rpc.batch.concurrency=64 --db.read.concurrency=64

Feel free to take a look at everything on GitHub! It’s all there waiting for you.

• Consensus/DA: When it comes to blob sidecars, just make sure to check in with the beacon REST using the block_id. If you're looking to take things up a notch, you can totally sync that data with S3 too! If you're looking for a handy resource, check out Coinbase/Base's blob-archiver. It's a solid open-source tool that you can definitely refer to for your needs! If you're looking for more info, check out QuickNode. They’ve got all the details you need!
• Solana: If you want to get your data flowing, definitely give Blockdaemon's AccountsDB→Kafka plugin a shot. You can also take a look at Solana’s handy Postgres plugin for some solid options! Partitioning by slot is a clever strategy. It really makes things easier when you need to rewind or reorganize stuff. If you're looking for more info, check out this GitHub page. It's got all the details you need!

3) Raw Zone and Lake/Warehouse

You can keep your raw JSON, SSZ, or Parquet files neatly organized based on the chain, whether it’s for execution or consensus, and by the day too. Just a simple way to keep everything in order! Don't forget to split things up by block_date and chain_id. This way, pruning will be smooth and efficient!

When diving into EVM, it’s helpful to get familiar with a few key concepts. Think of blocks, transactions, receipts, logs, traces, and if you want to dig a little deeper, optional state_diffs. You'll see these pop up quite a bit! Hey, just a quick reminder to snag those “builders/relays” whenever you have the chance using the MEV-Boost relay APIs. They're super helpful! If you're in a hurry and need a quick setup, definitely take a look at the Blockchain-ETL tools and schemas. They can really help you out! You can check it out here.

4) Transformations and semantic models

Stick with dbt, regardless of the warehouse you’ve got in play. It’s great for whipping up tested, incremental models. If managing ingestion infrastructure isn't really your cup of tea, you should definitely take a look at Dune's Datashare. It's a great alternative! It sends out carefully selected raw and processed tables directly to Snowflake, BigQuery, or Databricks. If your team is already using Snowflake, you really can’t beat Flipside’s zero-copy Shares as a great option. (dune.com).

5) Serving and BI

• Keep it simple with two main access points: use the “fast” serving tables for your dashboards when you need quick insights, and go for the “deep” tables when you’re diving into more detailed stuff like traces, blobs, and geyser streams. You can easily connect your notebooks or BI tools using Trino, or if you prefer, you can also use any connectors that are compatible with Trino. Another option is to stick with the native drivers for your data warehouse. For all the juicy details, just hop over here!

---

Exact configurations and APIs you will actually use

Ethereum execution: traces that scale

Reth offers a cool feature: it supports both Geth-style debug_* endpoints and Parity-style trace_* endpoints. This makes it pretty versatile! If you’re dealing with mixed tracer parity and need something quick, Reth is definitely the way to go! Also, it makes sure that the parity outputs stay consistent no matter which client you're using in your code. Hey, don’t forget to take a look at these important endpoints: debug_traceTransaction, debug_traceCall, trace_filter, and trace_replayTransaction. They’re pretty essential! If you want to dive deeper, you can check out more details right here. Erigon comes with a super useful trace module that includes endpoints like trace_filter, trace_block, trace_get, and trace_transaction. If you're dealing with range queries for addresses, this is definitely worth checking out. It's especially good at handling that kind of stuff! If you're looking for more details, check out the docs right here. They’ve got a ton of info that’ll help you out! Hey there, Python enthusiasts! If you're diving into coding with Python, I've got some great news for you. The web3.py library now has built-in support for trace_* calls with Erigon and Nethermind, making your life a lot easier. It's all set up and ready to go--no extra effort needed! Enjoy coding! Check it out here!.

Example: Getting All Traces Related to a DeFi Router Within a Block Window

So, if you want to find all the traces linked to a specific DeFi router in a particular block range, here’s a quick guide on how to get that done:

# Define the block range and the DeFi router address
start_block = 1234567
end_block = 1234590
router_address = '0xYourDeFiRouterAddressHere'

# Fetch the traces
traces = fetch_traces(block_range=(start_block, end_block), address=router_address)

# Process and display the results
for trace in traces:
    print(trace)

Key Points to Remember

Just a heads up--don't forget to swap out 0xYourDeFiRouterAddressHere with the real address of the DeFi router you want to check out! Make sure you've got the fetch_traces function set up right so it can link up with your blockchain data source.

• Go ahead and tweak the start_block and end_block variables so they match the block window you want to focus on.

With this, you'll get a handy list of all the interactions that happened with your selected DeFi router across the blocks you've picked. It makes tracking things super easy! Happy querying!.

{"method":"trace_filter","params":[
  {"fromBlock":"0x16F14E0","toBlock":"0x16F9FA0",
   "toAddress":["0xE592427A0AEce92De3Edee1F18E0157C05861564"]}],
 "id":1,"jsonrpc":"2.0"}

alchemy.com

Ethereum consensus: blob sidecars are part of your truth set

If you want to grab some blobs, just head over to /eth/v1/beacon/blob_sidecars/{block_id}. Once you’ve got them, you can stash them away in S3 or on your NVMe drives. Happy storing! Hey, just wanted to give you a quick heads-up! EIP-4844 requires that clients hold onto those blob sidecars for roughly 18 days, which translates to about 4096 epochs. Hey, just a heads up! If you don’t keep track of these, you could end up with some incomplete historical L2 batch payloads when you're trying to put together those L2 blocks. It’s definitely something to keep in mind! Take a look at this link: (eips.ethereum.org). You'll find some interesting stuff there! Hey, just a quick reminder to check out the Deneb spec when you're validating those inclusion proofs and versioned hashes. It's crucial to keep in mind that the consensus layer is all about data availability, while the execution layer has a different focus. If you want to explore this topic more, check it out right here: (ethereum.github.io).

• If starting from the ground up isn’t really your thing, Base’s blob-archiver is a great option. It provides a solid blueprint that can help you set up a long-term storage solution and an API layer for your sidecars. Pretty handy, right? Hey, why not take a look at it on GitHub? You can find it right here: github.com.

OP Stack rollups: fault‑proofs, challenge windows, and finality

So, let's talk about model withdrawal latency. The OP Stack has this really neat seven-day challenge window. It works in a couple of steps: first, you prove your claim, then you wait a bit, and finally, you get to finalize it. Pretty cool, right? Just a heads up--keep an eye on your risk dashboards for those key dates: “proven_at,” “challenge_expires_at,” and “finalized_at.” You don't want to miss any important updates! "Hey, if you want the scoop on the details, you can find them here. Give it a look!"

So, just a heads up: Fault proofs are set to launch on OP Mainnet starting June 10, 2024, kicking things off with Stage 1. Exciting times ahead! Hey, just wanted to give you a quick heads up! The Security Council is around as a backup, and that’s really something to keep in mind for your risk notes. It’s pretty crucial! If you're curious and want to learn more, you can check it out here. It's a great resource that dives into the details!

Arbitrum: post‑4844 reality

Hey, guess what? ArbOS 20 “Atlas” just rolled out, and now Arbitrum chains can totally handle 4844! Super exciting stuff! If you're into batch posting, you might want to consider submitting blobs instead of calldata. Not only is it a smart move, but it can also save you some money! Hey there! If you're working with Nitro infrastructure or getting into analytics, just a quick tip: double-check that your DA consumer is good to go with both calldata and blob modes. It’ll save you some hassle down the line! If you want to dive into the details, just click here. You'll find all the info you need! So, if you check out the Arbitrum documentation, you'll see that in Rollup mode, the data can be either calldata or blobs. On the flip side, if we’re talking about AnyTrust mode, it’s all about those DACerts--just a heads up to index those separately! If you're looking for more details, just check it out here.

Solana: Geyser streaming and state compression

Hey there! If you want to kick things off with streaming your account transactions and slot statuses, make sure to grab the Geyser plugin. You can easily send that data over to Kafka or Postgres--your choice! Once you’ve tackled that, you can dive into decoding those specific events related to the program later on. Hey, take a look at the details over here: (docs.solanalabs.com). You might find something interesting! So, when you're dealing with compressed assets, just remember to use Read APIs to index them. And don’t forget to double-check everything with Merkle proofs--you’ll definitely want to make sure that the tree root is on-chain. Hey, just a quick reminder to make sure you’re keeping an eye on these details: the asset ID, proof nodes, tree address, and canopy depth. It’s all important stuff! This will help you make sure your transactions are legit and also collect useful analytics. If you're looking for more details, check out this link: solana.com. You'll find a ton of useful info there!

Minimal Geyser→Kafka Config Snippet:

Hey there! If you’re diving into Geyser and Kafka, here’s a quick config snippet to help you kick things off. It should give you a solid starting point! This setup is going to be your go-to for connecting your Minecraft server to a Kafka instance.

kafka:
  enabled: true
  bootstrap.servers: "localhost:9092"
  topic: "minecraft-events"
  key.serializer: "org.apache.kafka.common.serialization.StringSerializer"
  value.serializer: "org.apache.kafka.common.serialization.StringSerializer"

Just throw this into your config file, and you should be good to go! Don’t forget to tweak the bootstrap.servers setting if your Kafka instance is running somewhere else. Happy gaming!.

{
  "libpath": "target/release/libsolana_accountsdb_plugin_kafka.so",
  "kafka": { "bootstrap.servers": "kafka-1:9092" },
  "filters":[{
    "publish_all_accounts": true,
    "transaction_topic":"solana.mainnet.transactions",
    "update_account_topic":"solana.mainnet.accounts",
    "slot_status_topic":"solana.mainnet.slots"
  }]
}

Check out the Blockdaemon Solana AccountsDB Plugin for Kafka on GitHub! It's a great resource if you're diving into the world of Solana and Kafka.

---

Schemas that won’t paint you into a corner

• EVM Star Schema
• Fact Tables: Let’s dive into our fact tables! Here’s what we’ve got:
• Transactions: sorted out by block_number and tx_hash.
• Logs: These include the tx_hash, log_index, and topic0. 3`.
• Traces: These are made up of trace_address[], the type, the action taken, and the result.
• Blob References: These are recorded using block_number, blob_idx, kzg_commitment, and versioned_hash.
• Dimensions: Here are the main dimensions we keep an eye on:
• Addresses: showing info like checksum, whether it's an EOA or contract, when we first spotted it, and the label_id.
• Contracts: This includes info on creation_tx, runtime_code_hash, and verified_source.
• Relays/Builders: These come with a name and an endpoint.
• Get Those Mapping Functions and Event Selectors Down Early: Seriously, mapping these out early on is a game changer! Check out the 4byte.directory API or the OpenChain successor backed by Sourcify to easily connect selectors with their signatures. It’s a reliable way to get the info you need! Make sure to keep an eye on collisions and how often they happen! Take a look at this: 4byte.directory. You might find it really useful!
• External Labels (with Lineage): Awesome news! Etherscan has just launched their v2 Nametag/Labels API and the CSV export feed. Very exciting stuff! Don’t forget to keep track of where you got your info and when you last updated it. It’s super important for maintaining a good audit trail! If you're looking for more details, check out the docs right here: docs.etherscan.io. It's got everything you need!

---

Transformations and modeling: keep it incremental

• If you want to keep those refresh costs down, think about using dbt incremental models that focus on (block_number, tx_hash) and (slot).
• Not feeling like dealing with your own ingestion setup? No problem! You can easily send those transformations right to where your data is already chilling out. Hey there! Just wanted to share that Dune Datashare has the awesome ability to send both raw and curated datasets directly into platforms like Snowflake, BigQuery, or Databricks. Pretty handy, right? Hey there! With Dune's new dbt Connector, you can totally run dbt "in place" right on Dune's warehouse. Plus, it even comes with Git-based continuous integration. Pretty cool, right? This is really convenient if you're looking for some well-organized DeFi or NFT models without the headache of handling the computing part yourself. (dune.com). Flipside’s Snowflake Shares let you enjoy free, zero-copy access across more than 30 to 35 different chains. This is great for businesses that want to standardize everything on Snowflake. (flipside-3887e776.mintlify.app).

---

MEV‑aware telemetry: builders, relays, blobs per block

Make sure you’re aware of who's working on your block, whether it's a relay or a builder, and check if the payload includes any blobs. Hey, don’t forget to include the columns for “builder,” “relay,” “num_blobs,” and “value_eth.” It’s super important to have those in there! "Feel free to check out the relay dashboards or APIs to double-check this info!" (relayscan.io). Hey, just a quick note: the PBS enshrinement isn't live yet. So, that means the mev-boost sidecars are still busy doing their thing in production. Having a go-to list of relays you trust is definitely smart. And don't forget to mix things up if you ever feel like you need a change! (ethereum.org).

---

Costing and sizing you can present to finance

Ethereum EL (Erigon v3.x + Caplin) Guidance for September 2025

• Storage Requirements:
• Minimal: Roughly 350 GB.
• Full: About 920 GB!
• Archive: Roughly 1. 77 TB. Hey there! So, let me tell you about the latest version, Erigon 3. It's pretty exciting stuff! Wow, version 1 really slashed those indexes! I mean, the Polygon archive is now a staggering 930 GB smaller than it was in version 3. That's impressive! Absolutely! If you're getting things set up, I'd definitely recommend going for NVMe SSDs. Aim for something in the 2 to 4 TB range - that way, you’ll have plenty of space to grow into and won't have to worry too much about reindexing down the line. Trust me, it’ll save you some headaches later! For more info, check out the Erigon docs. They’ve got all the details you need!
• Bandwidth Needs:
  If you’re running a non-staking node, you’ll want to shoot for a bandwidth of at least 5 to 25 Mbps. That should keep things running smoothly! If you're aiming for a quick sync and need to do some tracing, you’ll definitely want to consider getting a bit more than just that. Hey there! If you're looking for the details on the hardware you'll need for Erigon, just take a look at the hardware requirements. It's all laid out for you!
• Blob Storage:
  Get ready for your blob storage to keep expanding! Setting up S3 lifecycle policies is a smart move! You can set it up to automatically transition your sidecar data from warm storage to Glacier after it’s been around for more than N days. It’s a great way to save on costs while keeping your data organized. Oh, and make sure you've got an API ready for any of those historical queries you might need! If you want to stay organized, the Base blob-archiver pattern is a great option. Check it out on GitHub when you get a chance!

---

1) Reconstructing an L2 batch from L1 using blobs

First things first, go ahead and snag those blob versioned hashes from the EL block for your batch posting transaction. Alright, next up, you'll need to grab the sidecars. You can do this by hitting the consensus REST endpoint: /eth/v1/beacon/blob_sidecars/{block_id}. Just plug in your block ID there, and you'll be good to go! Don't forget to double-check those KZG commitments with the versioned hashes you have on hand. Once that's sorted out, go ahead and save the blob body along with its index. If you want to dive deeper into this topic, you can find more info here. It’s a great resource!

• Once you're done with that, the next step is to break down the rollup-specific batch format (like Arbitrum Nitro) into those L2 transaction frames. Just a quick reminder to make sure you add those L2 receipts from your L2 node. It's super important! Just a heads up--make sure your consumers are all set to deal with both calldata and blob modes after Dencun rolls out. If you're looking for more information, just head over to the documentation. It'll give you all the details you need!

2) Solana cNFT Lifecycle Analytics

First things first, let’s set up streaming for the Geyser account updates and feed them into Kafka. This way, we can whip up an "asset_changes" table. Sounds good? Alright, here's the plan: for each of those compressed assets, we need to pull the Read API proof. After that, we’ll compare it with the Merkle tree root that’s on-chain. Once we have that sorted, we can lock in the “cnft_current_owner” by adding in the proof metadata--things like depth and canopy. Easy enough, right? (solana.com).

3) EVM Signature Enrichment with Collision Handling

• Whenever you have a log for topic0 along with an input selector, just grab the data from the 4byte/Sourcify dataset. If you come across a bunch of candidates, be sure to jot them all down along with a frequency_rank. And don't forget to mark any that seem a bit unclear with ambiguous=true. Next, you'll want to come up with a new score using trusted sources from Sourcify, but only when those sources are available. Take a look at it here: docs.sourcify.dev. You won't want to miss this!

1. OP Stack Withdrawal Risk Panel.

This panel looks at the potential risks when it comes to withdrawing from the OP Stack. It’s basically a way to assess how safe the process is and what you might want to keep in mind when making those withdrawals.

• Set up a system that monitors OptimismPortal.proven events on Layer 1. So, we need to set up the challenge_expires_at to be proven_at + 7 days. After that, we’ll just need to make sure to confirm the finalization whenever the finalize events come up. Hey there! Just a quick note: make sure to set up alerts for when a big withdrawal is about T hours from expiring and there aren't any disputes happening. It'll help you stay on top of things! (specs.optimism.io).

---

Emerging best practices we recommend in 2025

If you want to maintain a “replayable” L2 history, don’t forget to archive those blobs that stretch past the roughly 18-day network window. If you're diving into Ethereum analytics, setting up sidecars to mirror your data to S3 using an easy API is a must these days. (eips.ethereum.org). You might want to think about trying out different EL clients. It can really help boost resilience and tackle those little differences in tracers. For instance, pairing Reth with Erigon could be a smart move! Hey, just a heads up! It’s definitely worth watching out for those tricky situations where the debug and prestate tracers don’t really line up. There are some known issues with that, so it's smart to stay on top of it. (github.com). Hey, just a heads-up--don’t overlook builder and relay metadata. It’s super important, so make it a priority! Even though PBS isn’t officially up and running just yet, the way builders are grouped together can really influence how we look at pricing and censorship. (ethereum.org). When you're mapping ABIs, make sure to use trustworthy selector sources and verified code. This will really help you steer clear of any mis-decoding headaches. Trust me, "selector collisions" can pop up, and you definitely want to avoid that! (docs.sourcify.dev). If you're working with Snowflake or BigQuery, you might want to check out some vendor-maintained feeds like Dune Datashare or Flipside Shares. They can be super helpful for those “coverage chains” where you really don't want to mess around with setting up your own infrastructure. Trust me, it’s a real time-saver! When it comes to in-house operations, focus on the parts that truly make your business unique, like your Layer 2 solutions or your NFT segment. Those are the areas that really set you apart! (dune.com).

---

Build vs. buy: a realistic split

• Build in-house when:
• It's super important to monitor retention, especially those blobs that seem to hang around for more than 18 days. If you're into custom tracers or if you need alerts that come through in under a minute, that's something to think about too!
• You're using some unique strategies, like MEV monitoring and keeping an eye on counter-party risk, that really need to stay within your VPC. Sharing this info isn’t a great idea!
• Buy/augment when: If you're in a rush to get some solid multi-chain coverage, you might want to check out Snowflake Shares through Flipside or Dune Datashare. Just keep in mind that you'll need to work within the vendor's service level agreements (SLAs). So, if you're into curated DeFi/NFT tables and want to use dbt without the headache of handling ingestion, the Dune dbt Connector is just what you need! (flipside-3887e776.mintlify.app).

---

Implementation checklist (30/60/90 days)

• Day 0-30 Alright, so first up, you’ll want to get Erigon running in full mode. Don’t forget to set up Reth for some debugging and tracing. Oh, and make sure you’ve got a consensus client ready to go as well. It’ll make everything smoother! Hey, just a quick reminder to turn on the rpcdaemon with trace/debug. You don’t want to miss that! Okay, so here’s what you need to do: First, go ahead and deploy the sidecar archiver using the Base blob-archiver fork to S3. Don’t forget to set up those lifecycle rules while you’re at it! It'll help keep everything organized and running smoothly. First off, let’s fire up the Solana validator with Geyser and Kafka. Once that's up and running, take a look at the topics and partitions to make sure everything's ticking along nicely. (docs.erigon.tech).
• Day 31-60 Hey there! First things first, let’s get those raw Parquet files into the data lake. Once that's set up, you can kick off your dbt project. Spend some time focusing on the essential models like tx, logs, traces, blob_refs, and cnft_assets. You've got this! Make sure to link up the selector/ABI lookup using 4byte or Sourcify. And don’t forget to pull in the labels from Etherscan. That way, you can easily keep tabs on where everything comes from and the timestamps! (docs.sourcify.dev).
• Day 61-90 Hey there! So, let's get into it. We need to combine the OP Stack with the Arbitrum DA parsers to handle both calldata and blobs. Sounds like a plan, right?
• Let’s set up telemetry ingestion for the MEV relay and builder, and then create some dashboards to help us visualize all that data!
• Set up alerts so we can keep an eye on the “finalized vs head” drift, check on any delays with the blob archiver, and track when challenge windows are about to expire. (relayscan.io).

---

Appendix: handy commands and queries

• Use the Erigon trace filter to target a specific range of addresses.

curl -s -X POST localhost:8545 \
 -H 'content-type: application/json' \
 --data '{"jsonrpc":"2.0","id":1,"method":"trace_filter","params":[
   {"fromBlock":"0x1700000","toBlock":"0x170FFFF",
    "fromAddress":["0x..."],"toAddress":["0x..."]}]}'
'

(alchemy.com)

• Beacon blobs sorted by head:

curl -s https://<your-beacon>/eth/v1/beacon/blob_sidecars/head

QuickNode

Hey, when you get a chance, check out the OP Stack withdrawal flow reference. It’s a great guide for organizing your events, and you can find all the details here: specs.optimism.io. Happy exploring!

Hey there! If you're thinking about getting the Arbitrum batch poster up and running for those 4844 blobs (just a heads up, this is specifically for Rollup mode), take a look at the “post‑4844‑blobs” settings. It’ll help you get everything sorted out! If you're looking for more info, check it out here. It's got all the details you need!

---

Final take

Looking ahead to 2025, it makes sense to go for a hybrid strategy. Hold on to the stuff that really reflects your style--think blob retention, chain-specific decoders, and MEV-aware telemetry. Meanwhile, for the rest, like wider multi-chain coverage, consider renting those out through Datashare or Shares. It's about finding that balance! With the setup we’ve put together--covering everything from configurations to endpoints and schemas--your team can get a really strong blockchain intelligence stack up and running in just three months. Plus, you can tweak and improve it as you go, especially with Ethereum gearing up for the SSF and PBS enshrinement. It’s all about feeling confident while you adapt! Take a look at this: (ethereum.org). You might find it pretty interesting!

---