MicroZed Chronicles: Local AI for RTL Generation

FPGA Horizons London- October 6th and 7th 2026 - get Tickets here.

The $99 Artix UltraScale+ Explorer Board - learn more here

One of the questions I get asked the most at the minute is whether AI can actually help us with our RTL development.

I have spoken several times about how frontier models can help us, and how our role as engineers is evolving as we are able to leverage higher levels of abstraction.

However, while frontier models are excellent, for many of us working in defence, space and other regulated domains there is an immediate complication, sending design data to a cloud API is simply not an option.

The interesting question then becomes whether a model running entirely on a machine on your own desk, with nothing leaving the building, is good enough to be useful. Rather than guess, I decided to measure it.

The Machine

The test machine is an AMD AI Halo box (Ryzen AI Max+ 395) which provides the Radeon 8060S integrated GPU and 128 GB of LPDDR5X shared between the CPU and GPU. I decided to purchase one when I was in the US recently.

The unified memory is the point here, there is no fixed VRAM limit as the GPU addresses the same pool as the processor. This means a model is not constrained by whatever memory happens to be soldered onto a graphics card.

Every model in this study, from 3.4 GB to 118 GB, ran fully GPU resident. Most of the models are served by Ollama over its OpenAI compatible endpoint, which means they plug straight into VS Code and command line tools.

A Little Theory: Tokens and the KV Cache

Language models generate text one token at a time, a token being a word fragment of a few characters. For each token generated the model's active weights must be read from memory, which makes generation speed a memory bandwidth problem and not a compute problem. Two consequences follow from this.

The first is that model size matters directly. A 4 bit quantised model needs a little over half a gigabyte per billion parameters, so a 12B model is about 7.6 GB and a 27B model is about 18 GB.

A dense model reads all of its parameters for every token, while a mixture of experts (MoE) model activates only a few experts per token and reads only those.

The clearest demonstration of this in my results is the dense Qwen3.8 27B which generated at 18.9 tokens per second, while the similarly sized Qwen3-Coder 30B MoE, which activates just 3.3B parameters per token, ran at 47.5 on identical hardware.

The second is that context is not free. The KV cache stores the attention state for every token in the context window, it is allocated up front and grows linearly with context length, coming out of the same memory pool as the weights.

I run a 16K window with the cache quantised to q8_0, which halves its footprint with no measurable effect on the results. The output budget matters as well, an early version of my benchmark capped generation at 1,200 tokens and the AXI task truncated for every model, 16,000 is the working figure.

The Method

I wrote a small Python harness which gives every model the same six design tasks. These are a parameterisable counter, a CDC synchroniser with ASYNC_REG, an AXI4-Lite slave, a UART transmitter, a synchronous FIFO with wrap bit pointers and inferred RAM, and a bug fix exercise. Each task exists in both VHDL-2008 and SystemVerilog.

The harness extracts the code from the response and compiles it, VHDL with GHDL and SystemVerilog with Icarus Verilog, while a keyword check confirms the model addressed the specification. If compilation fails the compiler errors are fed back for exactly one repair attempt, mirroring how these tools are really used. Generation ran at a temperature of 0.2 with thinking disabled, with three runs per task. This gives eighteen attempts per model per language, with an evidence manifest hashing every input so the results are reproducible.

Seven models took part, Gemma 4 12B, Qwen 3.5 at 9B and 4B, Qwen3-Coder 30B (MoE), Qwen3.8 27B (dense), bluepearl-rtl which is a domain tuned RTL model from Blue Pearl, and the 125B MoE Qwen3.8-Flash-Next which runs fully resident under a gfx1151 native engine with speculative decoding.

The Results

Compile passes out of 18 attempts, first try and after the single repair:

Several things stand out. Capability splits by size and tuning: the small models compiled 3 to 5 of 18 VHDL attempts first try, with conceptual errors like port maps inside entity declarations; the mid and large models managed 9 to 12; and the domain-tuned bluepearl-rtl topped both languages, the only model to pass all 36 attempts once the repair loop is included.

Flash-Next, the 125B, was the strongest general-purpose model: a clean 18 of 18 first try in SystemVerilog and 12 of 18 in VHDL, and despite being the largest model in the study it generated at 47.8 tokens per second, faster than models a quarter of its size, because only 6B of its parameters are active per token and the engine adds speculative decoding.

Its one cost is latency: each response opens with a reasoning preamble of a few hundred tokens, which pushes time to first token to around seven seconds where every Ollama-served model started in under a second.

Every model scored higher in SystemVerilog than VHDL, which tells you that Verilog-centric public benchmarks will not predict VHDL performance. The repair loop earned its keep everywhere, adding two to six passes per model for one extra generation.

The FIFO was the great separator: one first-try pass in 21 VHDL attempts across the whole field. And running each task three times showed the small models flipping between pass and fail on identical prompts, occasionally running away to the token ceiling, so never trust a single-run number.

Counting every generated token, including failed attempts, also inverts the economics: the accurate models cost around 1,200 tokens per passing VHDL module, while the 12B cost over 10,000, because most of its output was spent on attempts that failed.

A compile pass means the code parses and elaborates, nothing more. Plenty of compile-clean output was code I would reject in review, and my wider work with AI-generated designs has shown that as models get better, their defects get quieter, surviving lint, simulation, and even hardware before a careful reviewer catches them against the standard.

Next steps are to create test benches for the RTL and lint the RTL produced to ensure it is actually functional and meets what has been asked by the model. This I will look at next week as it will be quite in depth.

however, from first appearances it looks like local generation is viable today, but the verification burden does not disappear with a better model, it moves.

FPGA Conference

FPGA Horizons London- October 6th and 7th 2026 - get Tickets here.

FPGA Journal

Read about cutting edge FPGA developments, in the FPGA Horizons Journal or contribute an article.

Workshops and Webinars:

If you enjoyed the blog why not take a look at the free webinars, workshops and training courses we have created over the years. Highlights include:

• Upcoming Webinars Timing, RTL Creation, FPGA Math and Mixed Signal

• Professional PYNQ Learn how to use PYNQ in your developments

• Introduction to Vivado learn how to use AMD Vivado

• Ultra96, MiniZed & ZU1 three day course looking at HW, SW and PetaLinux

• Arty Z7-20 Class looking at HW, SW and PetaLinux

• Mastering MicroBlaze  learn how to create MicroBlaze solutions

• HLS Hero Workshop  learn how to create High Level Synthesis based solutions

• Perfecting Petalinux  learn how to create and work with PetaLinux OS

Boards

Get an Adiuvo development board:

• Adiuvo Embedded System Development board - Embedded System Development Board

• Adiuvo Embedded System Tile - Low Risk way to add a FPGA to your design.

• SpaceWire CODEC - SpaceWire CODEC, digital download, AXIS Interfaces

• SpaceWire RMAP Initiator - SpaceWire RMAP Initiator,  digital download, AXIS & AXI4 Interfaces

• SpaceWire RMAP Target - SpaceWire Target, digital download, AXI4 and AXIS Interfaces

• Other Adiuvo Boards & Projects.

Embedded System Book   

Do you want to know more about designing embedded systems from scratch? Check out our book on creating embedded systems. This book will walk you through all the stages of requirements, architecture, component selection, schematics, layout, and FPGA / software design. We designed and manufactured the board at the heart of the book! The schematics and layout are available in Altium here.  Learn more about the board (see previous blogs on Bring up, DDR validation, USB, Sensors) and view the schematics here.

All words in this blog were written by a human.