Open Source Support
Tool maintenance involves three areas:
- Ensuring that the tool works correctly in your environment.
- Maintaining existing customizations for your environment and adding further extensions as required.
- Keeping tools synchronized with their upstream code base.
Testing is central to all that we do. Indeed, the first thing we will do when taking over maintenance of any tool is to set up an automated nightly regression test system. Very often this brings to light errors that have been previously left unresolved, and fixing these becomes our priority.
Testing customization is an important aspect. If a tool has been modified for your environment, it is not sufficient to simply use standard regression tests and tests must be added for customizations.
Regression tests ensure that any problem, once fixed, should not recur. However, it is purely reactive. In addition we will provide regular testing — based on your typical usage — to proactively seek out problems before they occur. This is something that we implement in partnership with you, the customer. Very often we cannot use your code to test, since it is confidential. But we can set up systems to make it easy for you to test with such code and report to us.
Any project needs a mechanism by which to track issues. Our preferred software is the open source issue management tool, Bugzilla, which provides a rich and configurable environment for tracking issues. This can be run as either a private database, or as a public server to which anyone can contribute. For tools that are already public, we can also use the upstream bug tracker if preferred.
We can also use a GitHub issue tracker. This is a much simpler system and provides less functionality for tracking and reporting. However, it can be very convenient when a project is hosted on GitHub.
You may wish your extensions and modifications to a tool to be contributed to the upstream project. Embecosm has extensive experience with public contribution, including agreements with the Free Software Foundation to contribute to GNU projects.
Upstream contribution can increase the credibility of your tools and also your organization. Typically, acceptance of contributions requires a high standard of code development and testing, which provides a public recognition to your end users that the tool is of a very high quality.
Confidentiality and Security
While we work on open source tools, they are often used for projects that must remain confidential. A common example is development of compiler tool chains for yet to be announced processors.
Embecosm is experienced in working in such environments. Code can be hosted on our own encrypted servers, which are provided by Bytemark Hosting in York, UK. These servers are also used to host secure bug trackers, secure IRC servers and secure collaborative workspaces (Wikis).
By default all email communication is signed and encrypted, with remote access only via key-based SSH.
As an alternative, we also host projects in private GitHub repositories. This provides a flexible way to restrict access to projects. However, access control is then with GitHub, rather than Embecosm.
LLVM Tool Chain Maintenance
LLVM is not a single tool, but a large collection of related tools. Embecosm is a major contributor to the project and one of the few independent LLVM development companies in the world. Our particular speciality is with non-standard architectures. These represent a challenge for LLVM, which is primarily architected to support large modern RISC processors. Embecosm are responsible for the AAP architecture. Designed specifically to stress test compiler tool chains, this is a 16/24-bit Harvard architecture with multiple address spaces and word-address code memory.
Embecosm also leads the world in testing of LLVM, an area where this particular tool chain is notably very weak. We solve this by using the GNU compiler regression tests, which comprise around 75,000 C tests and 50,000 C++ tests. Our open source test harness allows easy configuration of the latest versions of these tests for any compiler and any target.
LLVM aims to provide a replacement for every component in the GNU tool chain. Many of those are relatively immature, so most practical LLVM tool chains use some GNU components.
The components which needs supporting are:
- The compiler. Comprising The C/C++ front end, Clang, the LLVM compiler itself and the integrated assembler. Version 3.7 is due for release in the second half of 2015, with new releases taking place approximately every year. Embecosm has expertise in all these areas, with Simon Cook authoring the guide to the LLVM assembler.
- The linker. The LLVM project has two linkers in development, lld and mclinker, but neither can be regarded as mature enough for general commercial deployment. Most LLVM tool chains use on of the GNU linkers: either gold, which is newer, but only supports ELF — or ld, which is older but more general purpose. Embecosm can support any of these.
- Core libraries. At the heart of LLVM is CompilerRT, providing low level emulation and built-in functions. Large performance improvements can be achieved by improving these, and Embecosm’s superoptimization expertise can achieve the best possible code. Supporting C++ also requires the Standard C++ library for LLVM, libc++.
- Standard C library. This provides the interface from C/C++ to real hardware. There are a number of C libraries available which are compatible with LLVM. They range from the relatively simple newlib — ideal for bare metal embedded targets — through to fully comprehensive libraries such as glibc, necessary to support Linux systems. Embecosm has expertise in all these, with one of our application notes being a standard guide to porting newlib.
- Debugger. LLVM now has its own source level debugger, LLDB, but it currently only supports ARM, x86 and Hexagon targets. Notably it has little support for embedded targets and as a result most tool chains use GDB. Embecosm has expertise in these, in particular for embedded targets. Jeremy Bennett is author of the standard guide to porting GDB.
- Low level utilities. LLVM has a growing collection of low level utilities, to match the tools provided by GNU binutils. Whether the LLVM or GNU versions are used, they form an important part of any tool chain.
GNU Tool Chain Maintenance
The GNU Compiler Collection (GCC) is not a single tool, but a large collection of related tools. Embecosm is a major contributor to GCC, being the official maintainer for the Epiphany and Synopsys DesignWare ARC implementations. We are one of a small number of independent GCC development companies in the world. Our particular speciality is with non-standard and embedded architectures.
GCC is now nearly 30 years old, and over that period has acquired a substantial body of regression tests. In the latest release there are around 75,000 C tests and 50,000 C++ tests. One of our key maintenance tasks is to ensure these regression tests are run automatically every night.
The components which needs supporting are:
- The compiler. The compiler provides support for a range of languages (C/C++, ObjectiveC/C++, Fortran, Modula 2, Ada). GCC 5.1 was released in April 2015, with major releases taking place approximately every year. Embecosm has expertise, particularly in back-end optimization, providing the official maintainer for the Epiphany and Synopsys architectures.
- The assembler. The compiler generates symbolic assembler as output, so a separate assembler is required to generate object files. This can either be handwritten, or generated with the assistance of CGEN. In the latter case, the same tool can also generate a disassembler and simulator for use either standalone or within the GNU debugger.
- The linker. The GNU linker, ld, supports a wide range of object file formats, but as a consequence has become very complex. More recent ports have used the ELF-only gold linker. Embecosm has the expertise to support either.
- Core libraries. At the heart of GCC is libgcc, providing low level emulation and built-in functions. Large performance gains can be achieved by improving these, and Embecosm’s superoptimization expertise can achieve the very best possible code. Supporting C++ also requires the Standard C++ library for GCC, libstdc++v3.
- Standard C library. This provides the interface from C/C++ to real hardware. There are a number of C libraries available which are compatible with GCC. They range from the relatively simple newlib — ideal for bare metal embedded targets — through to fully comprehensive libraries such as glibc, necessary to support Linux systems. Embecosm has expertise in all these, with one of our application notes being a standard guide to porting newlib.
- Debugger. The GNU Debugger has been around almost as long as GCC and provides comprehensive source code debugging. There is a standard interface to Eclipse, and the debugger can be integrated with a simulator. The simulator can be hand-written, created by CGEN or a similar tool, or be generated from a silicon chip design by using a tool such as Verilator. GDB 7.9.1 was released in May 2015, with major releases occurring approximately annually. Embecosm has made significant contributions to GDB, particularly in supporting Harvard architectures with multiple address spaces. Jeremy Bennett is author of the standard guide to porting GDB.
- Low level utilities. The GNU binutils provide a collection of low level utilities and form an important part of any tool chain. They are generally stable and simple to maintain. There is a new release approximately annually, but the changes are relatively modest and mostly concern support for new architectures.
Verilator is an open source tool which generates very high performance cycle accurate models from Verilog HDL chip designs. For many years, Embecosm has contributed to the development of this tool, both with performance enhancements and by adding functionality.
The free licensing model makes Verilator a very cost effective solution for silicon chip regression testing. With a rack of low cost Linux servers, nightly testing of large complex chips is feasible, increasing the quality of the design.
The models can also be provided early to the software development team, allowing early implementation of system software. Indeed, the the GCC implementation for the Adapteva Epiphany processor was completed by Embecosm before silicon was even taped out, using Verilator models of the hardware.
It is in the nature of Verilog that all new designs tend to expose bugs in tools, and although it has benefited from nearly 20 years of development, new designs can still trigger new bugs. It is also true that even the performance of Verilator generated models can be insufficient, and some hand- crafting of critical sections is needed.
If you are considering using Verilator in your silicon chip design flow, Embecosm can provide the tool support you need to minimize the risks associated with adopting the tool. And at a far lower cost than licensing proprietary simulation tools.