Feed on

Archive for July, 2007

Checks or Functional Coverage (Part II)?

Tuesday, July 24th, 2007 by David Robinson

[NOTE: This entry was written in response to some comments posted to my previous entry "Checks or Functional Coverage?". It was only meant to be a couple of line, but got a bit out of hand :-) ]

In my previous entry "Checks or Functional Coverage?", I made the point that checkers were more important than functional coverage, and that you had to get the checkers done first. Some of the replies said "I agree, but…" and then went on to say that we needed both. I completely agree; the ideal testbench will have checkers and functional coverage. My message here isn’t "do checks and forget about functional coverage". It’s "do the checks for the important requirements before the functional coverage for the important requirements, but do both of these before starting work on the less important requirements, because you might not end up with the time to do it all". That’s not very snappy though, so let’s go with "do the checks first".


Checks or Functional Coverage?

Monday, July 9th, 2007 by David Robinson

31 July 2007 - Fixed a typo.

Why does no one mention checkers any more? All I ever seem to hear is “functional coverage”, “functional coverage”, and more “functional coverage”. It appears that the entire verification industry is in the midsts of a functional coverage love-in that, while might be good for tool sales, isn’t very good for some verification teams.

The historical reasons for this are clear - EDA vendors had to sell new tools, so they went on a functional coverage marketing campaign. They had nothing really new to add to checking, but they sure had those fancy constraint solvers with functional coverage engines to sell. And slowly but surely, functional coverage took centre stage in everyone’s minds.

But it has gone too far. Functional coverage has become such a central pillar of verification that we’ve encountered teams who can tell us in gory detail what they have covered, but can’t tell us what they have checked. In one case, they hadn’t actually checked anything, although they did have 100% functional coverage (which turned out to be wrong anyway).

A quick look at the SystemVerilog LRM suggests that the checking requirement seems to have escaped the language designers as well. Sure, SVA is wonderful, but assertions only go so far towards checking a design (and not really that far when you think about it). What about support for all those higher level checks? Where are the language constructs for checking behaviour and reporting errors in the testbench part of SV? “if()” and “$display()”? Is that really it? That’s not what I was expecting from a language that has been designed for verification.

The functional coverage mantra is so engrained in the verification industry’s psyche that even non-tool vendors are preaching it. Let me quote from Janick Bergeron’s [1] "Writing Testbenches using SystemVerilog":

"Start with functional coverage …Thus, it is important to implement functional coverage models and collect functional coverage measurements right from the start".

He is not alone - I just happened to have his book to hand. Surely it should be something like “Start with checks. Who cares what the functional coverage is if you don’t have any checks? Who cares what the functional coverage says when your implementation metric is only sitting at 10% (e.g. only 10% of testbench code written)?”.

Experienced guys like Janick know that the checks have to be in place, but even the mention of checkers has faded so far into the background that some verification engineers don’t seem to know about them at all.

So what should you really do when writing your testbench?

  • Select your most important verification requirements. Pick the ones you absolutely have to get done
  • Write some stimuli for them
  • Write some checkers and check them
  • Once you get close to finishing the implementation that you have planned, put the functional coverage in
  • Repeat, but for your less important verification requirements

The point where you start concerning yourself with functional coverage is the point where you start going from the implementation phase (typing in the testbench code) to the closure phase (running tests and debugging). Now sure, I know they overlap quite a lot, but the point is that you get the checks in first because they are important. Functional coverage is a metric - passing checks are a necessity.

Look at it this way - if you had to run a testbench that had checks but no functional coverage, or a testbench that had functional coverage but no checks, which would be better?

Checks - no question about it.

So functional coverage might be the icing on the cake, but it will never be the cake. Checkers are the cake. You have to get the checks in first.


[1] Ok, he works for Synopsys, but his testbench books are neutral and generic. Buy yourself a copy - you won’t regret it

Work For Verilab