First serious RFS job - rfs bitmaps

Hi. Today entry is about bitmaps. RFS uses it to manage informations about blocks and inodes - are they free or not. This was my first task on RFS, fortunately there was big piece of test code on bitmaps written by  Chris Johns and I simply have to change it to fstest and extend it to cover rest of code.

My part of test contain check of set, clear and test map's bit when specified bit number is larger than map size. Another thing was use of rtems_rfs_bitmap_map_set_all function and then clear and set random bit. I have also to  attempt to find clear bit when all bits were set. I also invoke rtems_rfs_bitmap_map_clear_all directive.

In many functions there were branches which can execute probably only when pointer to bitmap control structure function was invalid, so I had to invoke all bitmap related directive with null pointer and check if return value is appropriate.

There were also another tricky and hard to execute branch in bitmap open directive. This branch could execute only when there would be lack of memory - it returned ENOMEM. It takes me a lot of time and I need help from my mentor - dr Joel Sherrill. He adviced me to use rtems_heap_greedy_allocate directive which cause allocation of almost all heap. You can specify how much blocks should be left on heap. I used this directive and it eventually helps. Attempt to open bitmap returned ENOMEM :)

All bitmaps code is covered. I think this is quite big success :) And my next battlefield will be block maps - which make use of bitmaps.

RFS bitmap test code is here.

fpathconf() test

As promised in last post, this entry is about fpathconf test. This is first test I was working on in range of developmental part of coverage. My mentor, Dr. Joel Sherril suggested me to take this function to cover first - as it should be quite simple to cover it. So I started my research. It gives not only full coverage of fpathconf but also discover some lacks in covoar - coverage tool in RTEMS coverage analysis.

RFS - my new battlefield

After long period of silence I am writing this post to summarize my last weeks efforts on RTEMS coverage.

Because coverage in core profile is near 100%, I and my mentor decided to start work on developmental parts of RTEMS. I looked on analysis results and tried to find out where are the largest uncovered ranges. My choice was RFS.

RFS is acronym of RTEMS File System. Some informations about this are here. Base filesystem in RTEMS is IMFS  = in memory file system. Other filesystems can be mounted in IMFS.

First of all I had learned how to mount filesystem and how to register RAM disk. This takes me a lot, I tried to understand everything, but there was still some errors and failures. Fortunately, this part of work on coverage of filesystems is done and you don't have to struggle with registering RAM and mounting filesystem on your own.

In directory rtems/testsuites/fstests there are set of tests of filesystems. There are also some support files, which allow you to focus on your unexecuted ranges and don't pay attention to rest. In directory rtems/testsuites/fstests/support there are files responsible for registration of RAM disk,  configuration of RTEMS, initialize file system. In directories named like FILESYSTEM_support (mrfs_support, imfs_support, mdosfs_support, etc.) are files related to specific file system. Thanks to these support files you have only write your test() function.

This approach allow to reuse test() in different file systems. You simply have to make directory mrfs_TESTNAME - for mounted RFS, mdosfs_TESTNAME - for mounted DOS file system, etc. and write makefile which takes appropriate support files. There are lot of tests in fstests and you can follow it to make your own filesystem test.

My next post would be about test on fpathconf() which also show some lacks in coverage tools. But it is another story :)

Second task - timespecs

Hi

I analyzed my coverage report and discovered that there is a lot of uncovered ranges in timespec math functions. I decided to go into it. I consulted it with dr Joel Sherrill and he advised me to write wrapper in RTEMS Classic API for these timespec functions as they are useful to users who do math on timespecs.

Maybe I start from begin :)
Timespec is struct defined in time.h header file. It is one of widely used elapsed time storage format. It is provided in GNU C Library as described here .
Timespec consist of two members:

long int tv_sec

This represents the number of whole seconds of elapsed time. 

long int tv_nsec

This is the rest of the elapsed time (a fraction of a second), represented as the number of nanoseconds. It is always less than one billion.

I look at other wrapper for score functionality available in Classic API - Chains. Looking at it I wrote my wrapper for all timespec related functions. I made them all inline, all names are now lower case and all starts from rtems_timespec.
After wrapper had been prepared I started writing new test hitting these functions. I named it sptimespec01 and it is now in rtems testsuite in sptests directory. I simply analyzed these routines code and do some math on timespecs to cause all branches were hitted and all instructions executed. It is working! This new test covered about 56 instructions. Quite good :)

All code including timespec sapi wrapper and new test can be found at rtems github or by simple click here

First task

Hi!

I'm here again. I would like to give a short note about my first task. I hope it could be interesting to someone :P

My first task in coverage improvement was concerned about small uncovered range in coremsg.c file from cpukit/score in rtems tree. In this file there is function which initialize message queue. Uncovered range was pointed to me by dr Joel Sherrill, my ESA SOCIS mentor. You can see this range at: uncovered range in annotated report from Joel's analysis

When I analyzed this annotated report it seems obvious for me that condition in if-statement
if(allocated_message_size < maximum_message_size)
is never true. I analyzed also math above it and it confused me a bit. It seemed to always increase value of allocated_message_size or at least ensure that it would be equal to maximum_message_size.

I shared with my observations with dr Joel and we agreed that if-statement can be removed. I prepared patch and posted it to rtems-devel mailing list. When I got first reply I decided to look this once again. I read comment and I understood that in one case allocated_message_size can be less than maximum_message_size even though math above appear to increase it value. When can it be? When overflow occur. 
According to this new look I revert my changes in coremsg.c and write new test - my first test :) - sp77. I attempt in it to cause overflow during creating message queue. It was very simple, I had to pass UINPTR_T-2 as maximum_message_size parameter in rtems_message_queue_create method.

Luckily, this was enough to cover completely coremsg.c :D First task passed!

P.S. My first test can be easily found at rtems github. It is direct link to it:
patch introducing sp77 test

Learnings:

• a lot! 1. how to add new test?
• how to run only one test for coverage purposes
• what is message queue :)
• be careful with overflow! 
• creating patches

How-to: prepare environment to run coverage analysis on RTEMS for pc386 BSP

I was trying to prepare environment to analyze coverage on RTEMS. Firstly I tried to do this for sparc/erc32 as it is most used by ESA (I’m ESA SOCIS participant). I tried to run do_coverage script or run_coverage script from rtems-4.11-work/rtems-testing/rtems-coverage/ directory. Unfortunately, scripts failed. I get message that script was unable to find tsim-erc32 on PATH. I started looking for it and I discovered that TSIM is commercial simulator for ERC32. Due to that I couldn’t get my environment ready to perform coverage analysis for ERC32. I knew that erc32 could be simulated in tsim or gdb-sis also, so I decided to try run do_coverage script with all steps (-A option when invoking do_coverage). And once again it failed. Sis is unsupported for coverage analysis.

As my efforts to have working environment for erc32 failed due to lack of TSIM I decided to prepare environment for another BSP. I chose i386/pc386. Although it is quite popular and I supposed to get much help I encourage quite a few problems while trying to get it working. One - missing dos2unix was very simple to solve.

yum install dos2unix

Another one was that couverture-qemu is developing and when trying to build it I get some warnings which end build process. Finally I solve this by disable breaking of compilation on compiler warning. I was also confused when after qemu installation I discovered that it also didn’t work because qemu executable changed name from qemu to qemu-system-i386. This problem is solved now and shouldn’t appear anymore if you have the newest rtems-testing git repository cloned and built. (I hadn't :-) )

To help anyone who would like to get this working I try to write some kind of tutorial.

TUTORIAL

To have rtems coverage analysis tools for i386/pc386 working you have to install qemu first. For coverage purposes it is recommended to install couverture-qemu as it is extended with trace generation possibilities.

You have to clone couverture-qemu git repository.

cd ~
git clone htpp://forge.open-do.org/anonscm/git/couverture-qemu/couverture-qemu.git
cd couverture-qemu/
git checkout origin/release-1.2.2
./configure --target-list=”i386-softmmu” --prefix=/home/rtems/qemu/install --disable-werror
make
make install

Now couverture-qemu should have been installed in /home/rtems/qemu/install directory.

To complete preparing qemu for using you should follow instructions from http://www.rtems.org/wiki/index.php/QEMU#Using_the_rtems-testing_Module
This include creating hd directory in /home/rtems/qemu/, cloning rtems-testing git repository, configuring and building rtems for i386/pc386.

It is also worth of remind that when working with rtems you should have some rtems path added to your PATH. On RTEMS CentOS Virtual Machine it is very simple:
. ~/rtems-4.11-work/setenv

After that you can try if it all works properly by running pc386 script with hello.exe.
There can also appear problems with path. If you get message “Unable to find qemu on PATH” you should export PATH to qemu installation.
export PATH = /home/rtems/qemu/install/bin/:$PATH

After all that steps, with pc386 BSP built and qemu installed you should be able to run coverage analysis succesfully.
When running coverage sripts you should be on the top of your RTEMS tree, in my case it is rtems-4.11-work directory.

The simplest form of running coverage analysis is running do_coverage script with all steps (cleaning rtems tree, configuring and building rtems, updating, copying and running tests, generating reports). It takes the greatest amount of time. If you had have configured and built RTEMS earlier you can skip some of this steps by putting some options when invoking do_coverage.

cd /home/rtems/rtems-4.11-work/
The simplest form of running coverage (all steps) for pc386 BSP
rtems-testing/rtems-coverage/do_coverage -A -B pc386

You can also specify other options in do_coverage scripts. To lists all this option type:
rtems-testing/rtems-coverage/do_coverage -?

Is it easy to understand? What should I add here else? I am waiting for your comments :-)

Hello world!

This is first post on this blog. I have just been accepted as student participant in ESA Summer of Code in Space 2012. I will be working on RTEMS test coverage improvement and this blog will be my worklog.

"RTEMS is a real-time executive which provides a high performance environment for embedded real-time applications including many features." As it is used in many safety critical applications like space projects it must be tested very well. Goal of my project is to improve test coverage of RTEMS, especially for SPARC/ERC32 BSP.