Leading the Embedded World

Modern Debug Tools for Linux

For engineers developing software for embedded Linux systems, MULTI for Linux brings the comprehensive debugging capabilities needed to dramatically improve productivity and bring a more reliable product to market faster.

Traditionally, Linux engineers have had to rely on debugging tools that offered only limited visibility and usefulness. GDB—and its related front-ends such as Eclipse—are cumbersome to set up, difficult to use, and not always trustworthy. As a result, developers often resort to the hit-and-miss effectiveness of printf or printk commands. Unfortunately, these are problematic as well and provide only a very small window into complex system execution.

With MULTI for Linux, embedded engineers can now visualize and debug all aspects of an embedded Linux system—including the Linux kernel, kernel threads, interrupt services routines, as well as user mode threads and processes—faster and more easily than ever before.

Debugging capabilities

MULTI for Linux offers a range of sophisticated debugging tools:

  • comprehensive Linux kernel visualization—including shared memory regions, semaphores, message queues, and kernel modules
  • support for SMP multicore debugging
  • debug from kernel through user space with one debugger
  • debug native Linux and virtualized Linux (on INTEGRITY Multivisor virtualization technology)
  • debug loadable kernel modules and device drivers
  • easily debug process startup code
  • set any thread-specific breakpoints in kernel code
  • debug shared objects in virtual address spaces
  • handle breakpoints with minimum performance cost
  • debug multi-threaded interactions across processes

System Requirements

  • Target processors: 32- and 64-bit Arm, Power Architecture, and x86 with SMP full support
  • Languages: C or C++
  • Compilers: Green Hills Linux compiler or GNU compiler
  • Target operating systems: any version of Linux
  • Hardware debuggers: Green Hills Probe V4

Run- and freeze-mode debugging

Application debugging with run-mode

Run-mode debugging of user-space applications gives programmers accurate and comprehensive visibility into real-world system behavior. To start debugging with MULTI, you simply load Linux and run your program. With just a few clicks, you can easily step through and browse even very large blocks of code with no lag time to:

  • view variables, data structures, and the call stack
  • view and modify memory locations
  • set hardware breakpoints
  • detect and load symbols for shared libraries
  • debug code right after a fork

Using run-mode debugging, Linux engineers can zero in on a specific area while the rest of the system keeps executing without disruption. You can halt, run, and set breakpoints on one specific thread—even if it uses code common to others—while the remaining threads continue to run.

The system information listed in the Linux Kernel Object Information window provides a detailed view of the complex internals of the Linux kernel as well as applications running on the system. Click for larger view

Complete system debug with freeze-mode

Freeze-mode debugging uses a probe or virtual machine to halt the entire system allowing MULTI’s OS awareness to provide a complete and detailed picture of the system as well as allow programmers to download, set breakpoints, step, run, and halt their program.

Freeze-mode debugging is particularly useful for:

  • debugging kernel panics
  • debugging device drivers and kernel modules
  • debugging interactions between user space and device drivers
  • debugging targets with no run-mode access, but with JTAG access
  • debugging complex interactions between a large number of system tasks
  • BSP development/early kernel startup
  • debugging user space processes early in boot up

More debugging features

Debugging on multicore processors

Green Hills Software was one of the first companies to offer a probe and debugger solution for multicore systems. MULTI for Linux includes features that make debugging multicore systems as straightforward as single core systems.

Key among these features is synchronous run control, which halts all cores as a unit when any core encounters a debugging condition. For instance, when a core hits a breakpoint the target list clearly shows:

  • which core(s) have hit the breakpoint
  • what thread was executing on the core at the time
  • what the other cores in the system were doing

Debugging virtualized Linux systems

Debugging is also straightforward on systems where Linux runs on a virtual machine (using INTEGRITY Multivisor). Traditionally, these kinds of systems would require multiple debuggers—a JTAG device alongside a console window to view print statements and GDB for the Linux machine.

But MULTI can debug everything from low-level INTEGRITY device drivers to Linux kernel drivers—virtualized or not—to applications running in Linux. No matter where your bug is, you will have full control and visibility to debug it.


Debugging kernel modules

Debugging in the Linux kernel space can be complicated—usually requiring special commands or GDB workarounds.

But MULTI’s comprehensive OS awareness enables straightforward debugging across the user space and into the kernel. Working in freeze-mode, MULTI can automatically determine which kernel modules are loaded to allow users to seamlessly debug device drivers and their associated user space applications.

MULTI for Linux can provide debug visibility throughout systems that combine Linux and INTEGRITY operating systems running in both native and virtualized modes.