From Blockchain Testing Framework to Production: Common Gaps and How to Close Them

Decision-makers usually think that having a solid unit test suite in Hardhat or Foundry means their project is all set for production. But let’s be real: the journey from local development to live networks is filled with potential pitfalls. We're talking about everything from protocol changes and fee dynamics to mempool behavior and those pesky L2 quirks, not to mention some operational gaps.

This guide points out the most common traps we encounter when rolling out blockchain software into production--and don’t worry, we’ll show you how to avoid them. We’ll share precise and up-to-date practices that take into account the changes happening across Ethereum L1 and the major L2s in 2024 and 2025.

Summary (meta description): Just because your tests pass in green doesn’t mean they’re good to go in production. Check out this expert guide that connects your Hardhat/Foundry tests to the real-world mainnet. We'll dive into testnets like Hoodi, Holesky, and Sepolia, discuss Dencun and EIP-4844 fees, explore L2 equivalence, MEV/privacy, upgrades, verification, and on-chain monitoring.

---

1) The “works on my testnet” fallacy: test networks keep changing

• So, it looks like Holesky is getting the boot after the Pectra test incidents. Don’t worry though, staking and validator lifecycle testing are being shifted over to the new Hoodi. Just a heads up, Holesky will officially stop being supported on September 30, 2025. Meanwhile, Sepolia is still your go-to for app and tooling testing, but it's set to retire on September 30, 2026. A replacement for Sepolia is on the horizon, scheduled for March 2026. For now, use Sepolia for your dapp testing and Hoodi for validator and staking flows. (blog.ethereum.org)
• In March 2025, the EF also decided to phase out the Mekong devnet. If you're developing apps or tooling, make sure to stick with Sepolia. For those of you practicing validator lifecycle rehearsals, check out Ephemery (which has weekly resets) or Hoodi. (blog.ethereum.org)

Practical Production Check

• Keep a testnet migration playbook handy in your repo, including chain IDs, faucets, RPCs, and any genesis/state assumptions.
• Set aside some engineering time every quarter specifically for testnet migrations and re-seeding fixtures.
• Make sure to pin your forked test state to a block number (check out section 3) to avoid getting tossed around by testnet changes.

---

2) Fees, data, and finality after Dencun: your cost and safety models changed

• So, Ethereum's Dencun (Deneb/Cancún) went live on March 13, 2024, and it rolled out EIP‑4844, which brings in these cool "blob-carrying transactions." What this means is that L2 data is now offloaded from calldata into short-lived blobs. The result? Big reductions in rollup fees and some new dynamics around "blob gas" supply and demand that you’ll want to factor into your models. Check out more on this here.
• Let’s break down EIP‑4844 a bit more: it's got a new blob transaction type (0x03), and the blob base fee is separate from the L1 base fee. Plus, these blobs get pruned after about 18 days and aren't directly readable by the EVM. Keep this in mind for your fee forecasts, simulation accuracy, and your analytics/receipts. You can dive deeper into the specifics here.
• Just a heads up: finality isn't instant on Ethereum. Currently, you hit economic finality after two epochs, which takes about 12.8-15 minutes. There’s also a chance of temporary finality loss during client incidents, so make sure to plan your operational confirmations accordingly. There's research on single-slot finality (SSF) aiming to tackle this, but we're not quite there yet. More details can be found here.

Practical Production Check:

• Keep an eye on L2 fee regressions after deploying. If your product depends on blob availability or calldata fallback, make sure to sound the alarm on any spikes in blob base fees.
• For those high-value operations--like bridges, upgrades, and payouts--it's better to wait for finality instead of just a set number of blocks. Don't forget to jot down the confirmation policies that are specific to your network!

---

3) Forked testing that actually matches mainnet

Local EVMs won't create the same kind of chaos you see on the mainnet unless you set them up that way.

• Both Hardhat Network and Anvil have got your back with mainnet forking! Just pin it to a specific block number and use an archive RPC to make your tests reliable and speedy (caching can boost your speeds by up to 20 times!). Check out the details here.
• The Hardhat docs lay it all out for you, covering everything from forking to chain configuration, plus they suggest block pinning. If you’re going off the beaten path from Ethereum L1, definitely use vendor archive nodes like Infura or Alchemy along with clear chain configs. More info is available here.
• Heads up if you’ve been using Tenderly forks in older setups! Tenderly officially ended support for Forks on March 31, 2025, so it’s time to transition to Tenderly Virtual TestNets/DevNets. These offer mainnet-data replicas, custom RPC methods, and handy tools for verification and debugging. Learn more about it here.

Example hardhat.config.js (forked, pinned):

require("@nomiclabs/hardhat-waffle");
require("@nomiclabs/hardhat-ethers");
require("hardhat-gas-reporter");
require("@nomiclabs/hardhat-etherscan");

// This is a sample Hardhat configuration file.
module.exports = {
  solidity: "0.8.4",
  defaultNetwork: "hardhat",
  networks: {
    hardhat: {
      forking: {
        url: "https://eth-mainnet.alchemyapi.io/v2/YOUR_ALCHEMY_KEY",
        blockNumber: 12345678, // replace with the block number you want to use for forking
      },
    },
    localhost: {
      url: "http://127.0.0.1:8545",
    },
  },
  etherscan: {
    apiKey: "YOUR_ETHERSCAN_API_KEY",
  },
  gasReporter: {
    enabled: true,
    outputFile: "gas-report.txt",
    noColors: true,
  },
};

Make sure to replace YOUR_ALCHEMY_KEY and YOUR_ETHERSCAN_API_KEY with your actual keys. Happy coding!

networks: {
  hardhat: {
    forking: {
      url: process.env.ALCHEMY_MAINNET_ARCHIVE!,
      blockNumber: 20123456
    }
  }
}

---

4) Choose maintained tooling--and wire it for CI

• In 2023, ConsenSys decided to bid farewell to Truffle and Ganache, archiving them since December 20, 2023, and teamed up with Hardhat for a smoother transition. If you’re still hanging onto Truffle or Ganache, it’s a good time to start planning your migration. Check out the full scoop here.
• Foundry has some cool features for stateful fuzz and invariant testing thanks to its “cheatcodes” (like vm.prank, vm.roll, and fork helpers). To get the most out of it, you’ll want to set up your invariant configs (think runs and depth) and remember to leave the cheatcode address out of your fuzz assumptions. For more details, take a look here.
• Certora has rolled out an alpha integration with Foundry that runs Prover checks on your tests, which can really level up your fuzz testing by bringing in formal guarantees. You can find more info on that here.

CI Add-ons That Close Gaps

• Gas Budgets: Check out the “forge snapshot” for easily tracking gas usage during tests and flagging any regressions. Plus, the hardhat-gas-reporter helps you keep an eye on costs. Both have been recently updated to play nice with broader multi-chain data and the Etherscan API V2. You can dive deeper here.
• Coverage: For ensuring that your critical paths are well-tested, you can rely on solidity-coverage or Hardhat’s built-in coverage flag. Just a heads up - it’s a good idea to skip gas-sensitive tests when running coverage. Get the lowdown here.

---

5) L2 ≠ L2: EVM “equivalence” vs. “compatibility” and what your tests miss

• The OP Stack, which includes Optimism and Base, aims for EVM equivalence by using the same state transition function as Ethereum, full precompiles, and Ethereum-like tools. But keep in mind that some parameters differ, like block times and EIP-1559 constants. These differences can really impact gas/price dynamics and timestamp logic, so it’s super important to ensure your test assumptions are chain-aware. More info can be found here.
• Arbitrum Nitro has done a great job of aligning its L2 gas semantics more closely with L1 and introduced some neat features like tracing and advanced calldata compression. This is fantastic for tooling, but developers still need to factor in the L1 data costs, often referred to as “2-D fees.” So, remember not to hardcode gas. Check out the full details here.
• ZKsync Era hit EVM bytecode compatibility in 2025 by rolling out an EVM Interpreter. This means you can deploy unmodified EVM bytecode using standard toolchains, which is pretty awesome! However, be aware of some known limitations, especially when it comes to debugging. Plus, using the interpreter mode could be 1.5 to 4 times more expensive than running on native EraVM. So, definitely validate your on-chain performance and plan for native compilers when you can. More info is available here.
• When it comes to precompile gaps, some L2-specific precompiles--for example, Arbitrum’s ArbSys--aren’t emulated by local forks. That means your integration tests need to hit the real testnet or specialized devnets to get accurate results. You can find more details here.

Practical Production Check:

• Make sure to parameterize your tests by chain, focusing on things like block time, fee elasticity, and base fee bounds. And hey, try to steer clear of any timestamp-sensitive logic if you can.
• Conduct a matrix CI by using this setup: {L1-fork, OP-stack testnet, Arbitrum testnet, ZKsync testnet} - all with the same specifications.

---

6) Oracles in reality: heartbeats, deviation thresholds, and decimals

Common bugs that pass unit tests:

• Assuming all Chainlink feeds use 8 decimals--that's a tricky one because not all do! Always make sure to pull the decimals() from the AggregatorV3Interface in your code. You can find more about it here.
• Treating oracle values as “streaming.” Keep in mind that feeds update based on deviation or a heartbeat, and those heartbeats can change depending on the network and asset. Just remember to check updatedAt and ensure you're enforcing freshness based on market risk. Check out the details here.

Safe Fetch with Freshness/Decimals

When working with data, ensuring that you're fetching it safely while also maintaining accuracy with decimals can be tricky. Here’s a quick guide to help you navigate this.

Why Freshness Matters

Freshness is crucial in data handling. It ensures that you're always working with the latest and most relevant information. When fetching data, you want to avoid stale or outdated responses that could mess up your calculations or decision-making process.

Handling Decimals with Care

When dealing with numerical values, especially decimals, precision is key. It's not just about fetching the data; it’s about ensuring that the values are accurate down to the last digit. Here are some tips to help you keep things precise:

• Use appropriate data types for storage and calculation.
• Employ rounding methods carefully to avoid unintentional errors.
• Regularly validate your data to ensure its integrity.

Example Code Snippet

Here's a simple code snippet that demonstrates how to fetch data safely while considering both freshness and decimal accuracy:

import requests

def safe_fetch(url):
    response = requests.get(url)
    if response.status_code == 200:
        return response.json()
    else:
        raise Exception("Failed to fetch data")

data = safe_fetch('https://api.example.com/data')

# Process the data ensuring decimal precision
def process_data(data):
    return round(float(data['value']), 2)

result = process_data(data)
print(result)

Conclusion

By focusing on both freshness and the careful handling of decimals, you can make sure that your data is not only current but also accurate. This approach helps reduce errors and improves the reliability of your applications.

(, int256 answer,, uint256 updatedAt,) = feed.latestRoundData();
require(block.timestamp - updatedAt <= MAX_STALENESS, "STALE");
uint8 dec = feed.decimals();
// Normalize to 18 decimals...

• When you calculate cross pairs (like BTC/EUR using BTC/USD and EUR/USD), make sure to account for slippage and set your tolerance levels for any stale feed issues. Check out the details here: (docs.chain.link)

---

7) Upgrades and proxies: catch the storage faults before mainnet

• Make sure to use OpenZeppelin Upgrades validation in your CI to catch any risky changes to your storage layout--like sticking with append-only structures, avoiding any reordering or renames, and remembering to use __gap for inheritance. Run oz validate and be sure to block any merges if there are layout diffs. (docs.openzeppelin.com)
• The plugins are designed to safeguard your upgrade flows (like deployProxy/upgradeProxy) by verifying compatibility with previous implementations. (github.com)
• Just a heads up: be careful when you’re switching major versions of OpenZeppelin Contracts between upgrades; the storage layouts can end up being incompatible when you leap across major versions. (forum.openzeppelin.com)

Important 2025 Change:

Hey folks! Just a heads-up: OpenZeppelin is rolling out a phased shutdown of their hosted Defender platform, with the final curtain coming down on July 1, 2026. If you're using Defender, like Relayers and Sentinels, make sure to carve out some time in your 2025-2026 plans to switch over to their open-source Relayer/Monitor stack or consider some alternatives. Check out the full announcement here.

---

8) Verification, provenance, and API changes you must plan for

• For keeping things steady over the long haul and syncing across multiple chains, it's best to go with Sourcify's byte-for-byte verification (that’s metadata-based, by the way). Make sure to integrate forge verify-contract or hardhat-verify along with metadata artifacts into your CI setup. Check out the details here: (docs.sourcify.dev).
• Heads up about the Etherscan API V2: mark your calendars for August 15, 2025, because that’s when you need to switch over (you’ll only need one API key for over 60 chain IDs!). It’s a good idea to update your explorers, reporters, and CI integrations now to dodge any issues down the line. More info can be found here: (info.etherscan.com).
• Just a quick note: the gas reporter will be moving to Etherscan API V2 in 2025, so double-check that your configurations are updated to reflect the new etherscan key field. You can find the release notes here: (github.com).

---

9) MEV, privacy, and mempool realism in staging and prod

Local tests don't really consider things like order flow, sandwiching, or private builders.

• Bring transaction simulation and private sending together in staging! With Tenderly simulations, you get detailed traces and state overrides on more than 100 networks. This is your chance to preview outcomes and gas costs before hitting that sign button. (docs.tenderly.co)
• Want to dodge public mempool front-running? Use Flashbots Protect RPC in production to make sure you’re only landing non-reverting transactions. You can customize how it behaves (like builders, refunds, mempool fallback, blockRange, and canRevert) through RPC query parameters. Pretty handy, right? (docs.flashbots.net)
• Heads up! Starting November 17, 2025, you’ll need signed headers for eth_sendPrivateTransaction. Make sure to update your clients to include the X‑Flashbots‑Signature and the new parameter formats. (collective.flashbots.net)
• If you're looking for enhanced private routing and observability, don’t forget to check out Blocknative’s Transaction Boost and mempool tools. They could really up your game! (docs.blocknative.com)

Practical Production Check

• For your staging pipelines, go ahead and simulate the bundle or transaction (that’s Trace + Gas). After that, submit it privately. Make sure to get an alert if there’s a chance the transaction could revert publicly or face a sandwich attack.
• Keep track of when to set “useMempool=true” as a backup for inclusion, especially if your private validators aren’t around. Check it out here: (docs.flashbots.net)

---

10) Runtime monitoring and threat intel

Shipping is just the halfway mark. When it comes to production, we really need to focus on real-time detection and response.

• Tenderly Alerts: You can set up over 12 different trigger types--like function calls, event parameters, failed transactions, state changes, and balance changes. Plus, you can choose where these alerts go, whether it’s Slack, Telegram, PagerDuty, or through webhooks. It’s super handy for spotting things like stuck queues, weird value transfers, or governance events. Check it out in more detail here.
• Forta Network: This is all about community-driven detection bots, including ones from Nethermind, the Forta Foundation, and various researchers. They help you keep an eye out for scams, phishing attempts, odd behaviors, and threats specific to different protocols. You can subscribe to alerts from these bots (like a spike in failed transactions or changes in ownership) and create your own allow/deny list policy. Learn more here.

---

11) Security testing that catches what unit tests miss

• Static analysis with Slither in CI: It's all about those speedy detectors and handy “printers” that help you wrap your head around inheritance and state writes/reads. Plus, there’s the new Slither‑MCP (coming in 2025) which fits right in with LLM-driven workflows and is still getting love from Trail of Bits. Check it out on GitHub.
• Property‑based fuzzing with Echidna: You can reuse properties for ERCs and run it alongside Foundry fuzzing and invariants. It's super cool! Dive into more details on this blog post.
• Symbolic testing with Halmos (a16z): This one’s all about taking your Foundry tests up a notch towards formal methods. It’s pretty neat! You can find it on GitHub.
• Certora Prover’s Foundry mode (alpha): This allows you to formally evaluate your Foundry tests, which is especially useful for those critical invariants. You can find more info in the docs.

Minimum Baseline

• Make sure to run Slither on every PR, and it should fail if there are any high-impact detectors.
• Introduce 3 to 5 protocol-level invariants, like preventing asset loss, ensuring conservation properties, and implementing bound checks. We need to enforce that invariant runs hit at least 1,000 times with a depth of 100 or more for each suite.
• Block merges unless core modules meet coverage thresholds, and don't forget to clearly mark any code that’s intentionally left untested.

---

12) A production‑oriented release checklist (plug‑and‑play)

Environment and State

• You can now run forked tests that are pinned to a specific block, plus utilizing archive RPC. Check it out here: (hardhat.org).
• We’ve got chain-parameterized tests for each target L2, which covers block time and EIP-1559 parameters. Take a look: (docs.world.org).
• Say goodbye to the old-school Forks! Tenderly DevNet/Virtual TestNet is here for your end-to-end rehearsals. More info can be found at: (docs.tenderly.co).

Fees and Finality

• Dencun-aware fee tests: These tests look into scenarios around blob base fees on Layer 2s and provide operational guidance for when fee spikes happen. You can read more about it here.
• Finality policy by network: Each network has its own finality policy. The ops scripts will patiently wait for finalization when dealing with high-value operations. Check out the details here.

Contracts and Upgrades

• Use oz-upgrades to “validate” in CI, check for differences in storage layouts, and block merges when there are layout issues. (docs.openzeppelin.com)
• Conduct an upgrade rehearsal from v1 to v2 on a forked state and run invariant tests across both versions.

Oracle Correctness

• Check out decimals(); to make sure we're keeping an eye on those staleness windows (you know, things like updatedAt/heartbeat). You can get more info over at (docs.chain.link).

Security/Testing

• We're using Slither, Echidna, and Foundry invariants in our CI processes. For those critical paths, we also rely on Halmos and Certora. Check it out here.
• To keep an eye on our gas budget, we take snapshots with forge snapshot and get cost visibility using hardhat‑gas‑reporter. You can find more info here.
• Finally, we run a coverage pass on our core modules. Take a look at it here.

Verification/Provenance

• We're rolling out Sourcify verification in CI! Plus, the Etherscan API V2 migration will be all wrapped up before August 15, 2025. Check out the details here.

Ops, Privacy, and Monitoring

• We're rolling out a private transaction path using Flashbots Protect RPC. Just a heads up, you'll need signed headers for eth_sendPrivateTransaction starting November 17, 2025. Check out the details over at Flashbots documentation.
• For alerts, we’re set up with Tenderly and Forta subscriptions. We’ve also got incident runbooks ready to go, complete with pause and timelock flows. If you want to dive deeper, take a look at the Tenderly alerts guide.
• We’re planning to migrate away from OpenZeppelin Defender by July 1, 2026. Instead, we’ll be moving towards an open-source Relayer/Monitor. Get more info on this transition in the OpenZeppelin blog.

---

13) Two practical examples

“Mainnet‑like Fork Test with Private Execution”

1. Start by forking the mainnet at a specific block using Hardhat or Anvil. Then, run a single simulation on Tenderly for the batch of transactions you plan to send. You can find more details here.
2. If your simulation goes through successfully, go ahead and submit it via the Flashbots Protect RPC with a 25-block window. If you hit a timeout, switch to the useMempool=true policy. Don’t forget to keep an eye on the status using the Protect Transaction API. Check out the docs here.

Upgradable Vault on OP Stack and Arbitrum

1. First off, run oz-upgrades validate on the builds for both chains. Make sure to stick to that append-only layout. You can check out the details here.
2. Next up, let's execute the invariant fuzz test on both L2 testnets. Don’t forget to add those chain-specific gas and timestamp assertions (remember, we’re working with 2-second blocks on OP chains). More info can be found here.
3. Finally, verify everything through Sourcify and Etherscan V2 just once, and make sure to propagate the metadata across the chain IDs. You can find more about that here.

---

14) What to budget for in 2025-2026

• Testnet Churn: So, we're moving from Holesky to Hoodi, and Sepolia is set to be swapped out in 2026. We should definitely build some migration playbooks into this process. (blog.ethereum.org)
• API Changes: Mark your calendars! Etherscan V2 will officially say goodbye on August 15, 2025. Time to update any explorers or reporters you’re using. (info.etherscan.com)
• Defender Sunset: We need to wrap up migration paths for Defender by July 1, 2026. Let’s make sure we’re prepared! (blog.openzeppelin.com)
• ZKsync: It's time to weigh the options between the interpreter route and going native for cost and performance. Plus, let’s keep an eye on how the toolchain matures. (docs.zksync.io)
• SSF Research: Stay in the loop, but let’s hold off on redesigning confirmations until we have clearer timelines for the mainnet. Good things come to those who wait! (ethereum.org)

---

Closing thought

A modern blockchain release process isn’t just about asking, “Did the tests pass?” anymore. It’s way more about figuring out, “Did we actually test the things that production handles differently?” By integrating forks, simulations, invariants, static/symbolic checks, private order-flow, and on-chain alerting into your workflow--and staying up to date with network and tooling changes--you'll dodge the priciest bugs: those pesky ones that only pop up on mainnet.

If you're in need of a production readiness review or a migration plan--like moving from Holesky to Hoodi/Sepolia, Etherscan V2, or managing a Defender exit--7Block Labs has got you covered. We can customize this checklist specifically for your stack and the chains you're targeting.