Thoughts on Verification: svlib - Including the Batteries for SystemVerilog (part 2)
In part 2, Alex and Jonathan continue covering features in svlib. In addition, they also cover how someone in the verification community can quickly ramp-up, request support for problems and get involved with svlib. Part 1 can be viewed here.
Alex Melikian: Another feature set offered by svlib in addition to what we’ve talked about so far, would be its functions related to the operating system. Once again, the standard SystemVerilog already offers some functionality interacting with the OS, but svlib takes it a level further. Can you tell us what svlib can do in this respect that SystemVerilog does not?
Jonathan Bromley: I can’t imagine a general-purpose programming language that doesn’t have library or built-in features for figuring out the time of day, discovering the values of environment variables, exploring the filesystem’s directory structure and so forth. We should expect that to be available, without fuss, in SystemVerilog - but, frustratingly, it is not. That’s precisely what svlib’s OS interaction features are intended to offer. There’s a collection of related sets of features - they’re distinct, but in practice you’ll probably use them together.
First, there’s a set of tools (implemented in the Pathname class) that allow you to manipulate Linux file and directory names in a convenient and robust way. If you try to do that as a simple string processing task, you’ll typically get some nasty surprises with things like doubled-up path separators (slashes); the Pathname class copes with all of that. Next, there’s a slew of functions for inquiring about the existence and state of files, mostly based on the “stat” system call that you may be familiar with. You can check whether a file exists, decide if it’s a directory or softlink, find whether you have write permission on it, and examine its creation and modification datestamp. You can also find what files exist at any given location, using the same “glob” syntax (star and query wildcards) that’s familiar to anyone who has used the “ls” command.
There’s easy access to system utilities for formatting a datestamp into a human-readable time and date, and that’s rounded out with functions to query the current wall-clock time of day - down to millisecond precision, if you need it.
And finally, you can query the existence and values of environment variables.
Everything I’ve listed there is fairly low-level stuff, and at a glance it may be difficult to see what verification tasks it supports. But it provides you with a very flexible set of tools to get information into and out of your testbench. To give one simple example, it means that for the very first time it’s really easy to add time and date information in the header of a log file you generate. How crazy is it that SystemVerilog has never had convenient facilities for doing that! Another example: your simulation can now, easily, use environment variables to control some aspects of its operation. And multiple simulation runs in the same work area can automatically generate new, unique names for log files if they discover that some files already exist as the result of earlier runs.
For many years, a handful of programmers who were expert in both C and Verilog have been able to use the VPI and DPI to interact with the operating system in the way I’ve described. svlib’s goal is to bring that capability within easy reach of everyone, with no need to write any C code of your own.
AM: I like the sound of that! Bringing functionality within easy reach is always what any developer is looking for in a library.
Now that we’re talking about ease of use, I want to bring up another point when I feel something should be deemed “easy to use”. That term, as we all know, has been a bit abused in the past, and in some cases does not hold true. For me, when I hear the term “easy to use”, it’s not only about how easy it would be to execute a feature, but also the level of ease in accessing critical information for the user if something goes wrong, or applied incorrectly.
In the case of using a third-party or external library, there’s always a lingering fear that something under the hood will go wrong, or the programmer will inadvertently misuse something. In such situations, not only does the desired action not occur, but there’s also no indication to what exactly has gone wrong. However, in the case of svlib, a lot of thought and effort has been put into making error reporting concise and easy to observe for the user. Can you give some detail to our readers on how svlib helps users in this department?
JB: This is another one of those areas where the eventual implementation was pretty straightforward, but figuring out what was really needed turned out to be surprisingly tricky. Unfortunately SystemVerilog lacks a “try/catch” exception handling construct that you find in Python, Java and many other modern languages. So we had to find a solution for ourselves.
Users have widely different needs for error handling and reporting. Examples are: someone doing some initial experiments to feel their way around svlib’s features, a programmer writing verification infrastructure that will be deployed and adapted by others, and a programmer who has good reason to be confident that his calls to svlib functions cannot possibly provoke an error - all these people will hope for different error-handling behaviors. And, of course, svlib has no way to know which kind of user is calling its functions!
An experimenter probably wants the library to report any problem automatically and very obviously, providing timely feedback on programming mistakes. In production code, though, you really don’t want a library to throw a bunch of diagnostic messages just because someone tried to query the creation date of a nonexistent file; instead, svlib should pass back an error code so that the calling program can handle it.
It’s also not obvious what is the best way to report a problem back to the caller. Our first attempt at this aimed for total consistency. Every library function - yes, really, every one - returned an error code as its result. In practice, trying to use that API was a clumsy nightmare. That taught me an important lesson: consistency is worth aiming for, but usability is way more important!
After a lot of thought and false starts, we settled on a design in which the user flips a switch to determine how errors are handled. By default, any error detected by the library will trigger an immediate assertion that reports the problem - the “experimenter” requirement. But you can call a mode-change function to request svlib to switch off this automatic reporting. It then becomes your responsibility to call an error check function after any error-prone svlib operation, such as regex processing or anything that accesses the file system. That function gives you an error code for the most recent operation, and you can also get a human-readable description of the error for use in your own messages. Finally, if you are 100% certain that your call to some svlib function can never cause an error, then you don’t need to do anything at all, and your code is not cluttered with any error handling whatever.
I know this sounds a bit complicated, but it seems to give a good balance between safety and flexibility. Many svlib functions simply can’t error, so in practice the error handling burden on a programmer is quite low.
There are some interesting subtleties around keeping error handling separate for each individual SystemVerilog process. I don’t want to go into the gory details here, but if you’re interested it’s all in the published material on our website.
AM: Thanks for summing that up, glad to see a healthy amount of thought was put into error reporting. It always helps to know there are methods available to give clues on what the root cause of a problem might be, especially when using third-party libraries.
Alright, I think we’ve yapped enough about svlib’s features, now lets tell our readers how they can put it to action! We’ve already indicated where they can find it, but what about getting their feet wet? Since you are the ‘horse’s mouth’ on this, how would you suggest is the best way for someone to ramp up on svlib?
JB: The download comes with a “doc” directory where you’ll find the user guide, and a copy of the DVCon-2014 paper and presentation. The presentation slides give a very quick walk-through of some example uses of svlib, showcasing the shape of the library and what it can do for you. The paper is, of course, quite a bit more formal; you probably want to look at that only if you’re interested in the decisions we had to take when designing svlib’s API and functionality. The user guide is where you need to go for details of all the features and how to use them. We’ve done our best to make it readable and accessible, but these things are inevitably subjective and we’re always glad to hear suggestions for how to improve the document.
As with many such things, the best approach is simply to try it. Right now, with a relatively small user base, it’s easy for us to answer the few questions that people send to svlib@verilab.com. Longer term, we plan to make use of bitbucket, github or some similar code collaboration tool to encourage and support a user community.
AM: Cool. It’s always preferable to get such instructions directly from one of the creators.
JB: It’s probably worth bearing in mind that “ramping-up on svlib” is a very different kind of thing than, let’s say, “ramping-up on the UVM”. A full-dress methodology like the UVM isn’t just a code library. Adopting it is supposed to change your whole way of working, to help you align your own work with industry best practice. By contrast, svlib has far more modest goals: it’s just a toolkit, to be cherry-picked as and when you think it’s useful, slotting into your existing code without any disruption. I really hope that many users will find - even if they don’t use svlib very often - that it allows them to do things, easily, that otherwise would be too much bother.
AM: I agree, I don’t think any of us wish to portray svlib as a ‘game changer’ in the world of verification. However, that’s not what it’s supposed to be. I think you and the others who have contributed to svlib’s development have done a good job of keeping it lightweight and fairly easy to learn, thus more adoptable.
As we’ve pointed out a few times in this discussion, though svlib provides a good set of features, it is very much a work in progress on certain fronts. You’ve mentioned where our readers can go if they have feedback, but what about bug reporting? Where can they turn to if they believe they’ve found a problem?
JB: Right now there’s a single point of contact for all svlib issues, whether they’re suggestions for improvement, bug reports or just “how do I get XYZ to work” questions: svlib@verilab.com. I appreciate that’s not very well structured, but the current level of activity is such that we’re able to cope with it and - so far, at least! - respond fairly promptly.
A number of people have suggested that we set up some kind of public repository on sourceforge, github, bitbucket or a similar platform. Verilab has already done that with some other projects and it’s definitely our plan to do the same with svlib. That will provide nicer reporting and collaboration tools.
This would be a great moment for me to thank the good folk who have already reported bugs - the latest release available on our website incorporates fixes for all of them.
AM: Yes, it’s important to mention that svlib cannot grow without the participation of the verification community along with their feedback and bug reporting. I’ll also remind our readers that they can be alerted of any updates to svlib through our twitter handle.
I think we’ve just about covered the basics, of course our readers can learn more through the resources we’ve stated. I know you and some of the others here at Verilab have done a solid job of putting it together and will continue to develop it to its full potential.
Jonathan, thanks again for your time, I’m already looking forward to our next encounter.
JB: Always a pleasure, Alex.
