Using ODB on Mobile and Embedded Systems

From Code Synthesis Wiki

(Difference between revisions)
Jump to: navigation, search
Revision as of 12:22, 21 February 2013
Boris (Talk | contribs)
(→Building the Example Using Static Libraries - Mention autotools option)
← Previous diff		 Current revision
Boris (Talk | contribs)
(Add link to Android guide)
Line 3: Line 3:
This guide discusses a number of topics relevant to using ODB on mobile and embedded systems. Specifically, it shows how to cross-compile the ODB runtime libraries in both static and shared variants. It also provides an indication of footprint as well as ways to minimize the generated code size. This guide discusses a number of topics relevant to using ODB on mobile and embedded systems. Specifically, it shows how to cross-compile the ODB runtime libraries in both static and shared variants. It also provides an indication of footprint as well as ways to minimize the generated code size.
-While this guide uses the [http://www.raspberrypi.org Raspberry Pi] ARM GNU/Linux computer as a sample target, other mobile/embedded platforms will normally require very similar steps. We will also explicitly mention a few areas that may need platform-specific changes. If you run into any problems while trying to apply the below steps for your platform, feel free to ask for help on the [http://www.codesynthesis.com/mailman/listinfo/odb-users odb-users mailing list].+While this guide uses the [http://www.raspberrypi.org Raspberry Pi] ARM GNU/Linux computer as a sample target, other mobile/embedded platforms will normally require very similar steps. We will also explicitly mention a few areas that may need platform-specific changes. If you run into any problems while trying to apply the below steps for your platform, feel free to ask for help on the [http://www.codesynthesis.com/mailman/listinfo/odb-users odb-users mailing list]. For an Android target, refer to [[Using ODB on Android]].
Some mobile and embedded targets support compilation on the target itself (the so-called self-hosted development). However, due to various constraints (RAM, CPU, connectivity, etc.), it is rarely practical to develop any real-world application on the target directly. As a result, a cross-compilation environment is normally used instead and in this guide we will only consider this approach. Some mobile and embedded targets support compilation on the target itself (the so-called self-hosted development). However, due to various constraints (RAM, CPU, connectivity, etc.), it is rarely practical to develop any real-world application on the target directly. As a result, a cross-compilation environment is normally used instead and in this guide we will only consider this approach.
Line 143: Line 143:
cd odb-examples-X.Y.Z/hello cd odb-examples-X.Y.Z/hello
-To build the example we will use modified manual build instructions from the accompanying <code>README</code> file. First, we compile the <code>person.hxx</code> header with the ODB compiler:+While it is possible to use autotools-based build system to cross-compile the examples, in this guide we will use modified manual build instructions from the accompanying <code>README</code> file. First, we compile the <code>person.hxx</code> header with the ODB compiler:
../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx ../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx

Current revision

Contents


This guide discusses a number of topics relevant to using ODB on mobile and embedded systems. Specifically, it shows how to cross-compile the ODB runtime libraries in both static and shared variants. It also provides an indication of footprint as well as ways to minimize the generated code size.

While this guide uses the Raspberry Pi ARM GNU/Linux computer as a sample target, other mobile/embedded platforms will normally require very similar steps. We will also explicitly mention a few areas that may need platform-specific changes. If you run into any problems while trying to apply the below steps for your platform, feel free to ask for help on the odb-users mailing list. For an Android target, refer to Using ODB on Android.

Some mobile and embedded targets support compilation on the target itself (the so-called self-hosted development). However, due to various constraints (RAM, CPU, connectivity, etc.), it is rarely practical to develop any real-world application on the target directly. As a result, a cross-compilation environment is normally used instead and in this guide we will only consider this approach.

Note also that this guide is not an introduction to developing C++ applications for mobile/embedded systems. In particular, it assumes that you are familiar with the concept of cross-compilation and how it works for your target. Before proceeding further with this guide, it is recommended that you install the cross-compiler for your target on your development machine, build a simple "Hello, World" C++ application, copy it on to your target, and make sure it runs correctly, that is, there are no missing libraries, crashes, etc.

In this guide we will use the arm-bcm2708hardfp-linux-gnueabi cross-compiler toolchain from the official Raspberry Pi tools repository. However, other toolchains can be used as well, such as the Debian/Ubuntu ARM cross-compiler package.

The most popular choice of a database for mobile/embedded systems is SQLite. So in this guide we will build the "Hello, World" example from the odb-examples package for the Raspberry Pi target using SQLite as the database. This, however, is not to say that other databases (or even multiple database at once) cannot be used on mobile/embedded systems. All we will need to do is cross-compile or get an already pre-built client library for the database(s) we want to use.

The following list provides a high-level overview of the steps we will need to perform in order to build an ODB-based application for our mobile/embedded target. The comments in brackets indicate whether this step is performed on the development machine or on the target.

  1. Install the ODB compiler [development machine]
  2. Cross-compile the database (client) library and ODB runtimes [development machine]
  3. Compile headers with the ODB compiler [development machine]
  4. Cross-compile the generated code and application code [development machine]
  5. Copy the application onto the target and run [target]

The easiest way to install the ODB compiler is to use the pre-compiled binary package for your development machine. For example, if your development machine is x86-64 GNU/Linux, then you will use the odb-X.Y.Z-x86_64-linux-gnu package. You can also build the ODB compiler from source code if you wish (note that in this case you will need to build it for your development machine, not for your mobile/embedded target).

The next step is to cross-compile the SQLite database (or another database client library) and the ODB runtimes libraries. Here we have two options: we can build them as static or as shared libraries. When it comes to mobile/embedded systems, static libraries have a number of advantages. Firstly, when using static libraries, the resulting executable will only contain ODB runtime code that is used by the application. This minimizes the executable size (both on disk and in RAM). Static libraries are also easier to work with; they are easier to build, to link to, and they don't need to be deployed to the target. Shared libraries in the mobile/embedded context may have an advantage if more than one application is using them. As a result, we recommend that you use static libraries unless you have multiple applications that use ODB and even in this case it makes sense to actually check if there are any footprint savings. If you decide to build and use shared libraries, we strongly recommend that you first try to build static variants to make sure that everything works in that simpler case.

Note also that it is possible to use a shared library for the database and static libraries for the ODB runtimes. This would make sense, for example, if SQLite came pre-installed (as a shared library) on your target because it is used by other applications.

As the first step, choose a working directory where we will build everything. Let's also create the install sub-directory in this working directory; this is the place where we will install headers, libraries, etc., that we are going to build (note that it's a bad idea to install them into, say, /usr/local because these libraries will be built for our target, not for the development machine).

Let's also assume that our cross-compiler toolchain and the ODB compiler are also in this working directory. That is, we have the arm-bcm2708hardfp-linux-gnueabi sub-directory (cross-compiler for Raspberry Pi) and the odb-X.Y.Z-x86_64-linux-gnu or similar sub-directory (ODB compiler). Let's also rename odb-X.Y.Z-x86_64-linux-gnu to just odb for easier referencing. Note that the cross-compiler and the ODB compiler don't really have to be in our working directory. If you have them installed somewhere else, simply adjust the paths in the instructions below.

Let's also add the cross-compiler to the PATH environment variable. That is, in a terminal where you will perform the compilation, do: 

   export PATH=`pwd`/arm-bcm2708hardfp-linux-gnueabi/bin:$PATH
   arm-bcm2708hardfp-linux-gnueabi-g++ --version

The second command verifies that the C++ cross-compiler can now be executed directly by printing its version.

Another step that we need to perform before we can start building is to verify that the cross-compiler toolchain doesn't have broken .la files, specifically libstdc++.la and, if you are using a pre-built SQLite or another database (client) library, the .la file for that library. The .la files often end up broken because of the different directories where the toolchain was initially installed (by whomever built it) and where you unpacked it. The .la files will almost always end up broken if you installed your toolchain by simply unpacking a .tar.gz archive. On the other hand, if you built the toolchain yourself (and didn't move the installation directory) or if you installed it from a package (e.g., from Debian/Ubuntu) then the .la files are most likely intact and you can skip this step.

To verify that the .la files are correct, search for libstdc++.la in the toolchain directory. If none is found, then you don't need to do anything. If one is found, open it in a text editor and search for the libdir variable (usually right at the bottom). The value of this variable should be the path to the directory where the libstdc++.la file resides. If it is not valid, correct it and save the file. If you are using a pre-built SQLite or another database (client) library, repeat these steps for its .la file.

The next section discusses building the database and the ODB runtimes as either static or shared libraries. The two sections after that show how to build and run the hello example using static or shared libraries, respectively. The final section in this guide provides an indication of the resulting binary sizes as well as covers various ways to minimize the generated code size.

[edit]

Building SQLite and ODB Runtime Libraries

Unless you already have SQLite built for your target, let's first do that. Download the sqlite-autoconf-XYZ.tar.gz archive and unpack it into the working directory: 

   tar xfz sqlite-autoconf-XYZ.tar.gz
   cd sqlite-autoconf-XYZ

Next we need to configure SQLite for our target. Here you may need to add additional C compiler flags that are specific to your target. For instance, if you are using a generic ARM toolchain (such as from from Debian/Ubuntu) instead of the Raspberry Pi-specific one, then you may need to specify the CPU version, etc., explicitly by adding the -march=armv6 -mfpu=vfp -mfloat-abi=hard options to the CFLAGS (and later CXXFLAGS) variable. You may also want to adjust the optimization level. Note also that the value of the --host option must match the cross-compiler tool prefix (that is arm-bcm2708hardfp-linux-gnueabi in arm-bcm2708hardfp-linux-gnueabi-g++) exactly. The basic configuration for a static SQLite library looks like this: 

   ./configure CFLAGS="-Os -DSQLITE_ENABLE_UNLOCK_NOTIFY=1" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

And for a shared SQLite library — like this: 

   ./configure CFLAGS="-Os -DSQLITE_ENABLE_UNLOCK_NOTIFY=1" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

If here or below a configure script terminates with an error, then you can find more detailed information about the cause of the error in the config.log file created by this script.

If, however, there are no errors during the configuration, then the next step is to build and install SQLite: 

   make
   make install

After this command you should have the SQLite headers and static/shared library in the install sub-directory of the working directory. Here and below you can also use the install-strip target to strip the installed libraries.

Once SQLite is ready, we can move on to building the ODB runtimes. The minimum that you will need is libodb and libodb-sqlite. If you also want to use one of the profile libraries, then you can build it in the same way.

To build libodb, unpack its source code into the working directory: 

   tar xfz libodb-X.Y.Z.tar.gz
   cd libodb-X.Y.Z

The configuration and building steps are similar to SQLite. Again, don't forget to add any extra compiler options to CXXFLAGS if your target requires them. Static library: 

   ./configure CXXFLAGS="-Os" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Shared library: 

   ./configure CXXFLAGS="-Os" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Once configuration is done: 

   make
   make install

To build libodb-sqlite, unpack its source code into the working directory: 

   tar xfz libodb-sqlite-X.Y.Z.tar.gz
   cd libodb-sqlite-X.Y.Z

Then configure and build. Static library: 

   ./configure CXXFLAGS="-Os" CPPFLAGS="-I`pwd`/../install/include" LDFLAGS="-L`pwd`/../install/lib" --disable-shared --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Shared library: 

   ./configure CXXFLAGS="-Os" CPPFLAGS="-I`pwd`/../install/include" LDFLAGS="-L`pwd`/../install/lib" --disable-static --host=arm-bcm2708hardfp-linux-gnueabi --prefix=`pwd`/../install

Once configuration is done: 

   make
   make install
[edit]

Building the Example Using Static Libraries

Once the static libraries for SQLite and the ODB runtimes are ready, we can build the hello example. First, unpack the odb-examples package: 

   tar xfz odb-examples-X.Y.Z.tar.gz
   cd odb-examples-X.Y.Z/hello

While it is possible to use autotools-based build system to cross-compile the examples, in this guide we will use modified manual build instructions from the accompanying README file. First, we compile the person.hxx header with the ODB compiler: 

    ../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx

Next we build everything with the cross-compiler: 

   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c person-odb.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c -DDATABASE_SQLITE driver.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -L../../install/lib -o driver driver.o person-odb.o -lodb-sqlite -lodb -lsqlite3 -lpthread -ldl

The result of the last command is the driver executable which we can copy over on to the target and run: 

   raspberrypi $ uname -a
   Linux raspberrypi 3.2.27+ #3 PREEMPT Sat Dec 15 18:52:34 SAST 2012 armv6l GNU/Linux
   
   raspberrypi $ ./driver --database /tmp/test.db
   Hello, John Doe!
   Hello, Jane Doe!
   
   count  : 3
   min age: 31
   max age: 33
[edit]

Building the Example Using Shared Libraries

Once the shared libraries for SQLite and the ODB runtimes are ready, we can build the hello example. First, unpack the odb-examples package: 

   tar xfz odb-examples-X.Y.Z.tar.gz
   cd odb-examples-X.Y.Z/hello

While it is possible to use autotools-based build system to cross-compile the examples, in this guide we will use modified manual build instructions from the accompanying README file. First, we compile the person.hxx header with the ODB compiler: 

    ../../odb/bin/odb -d sqlite --generate-query --generate-schema person.hxx

Next we build everything with the cross-compiler: 

   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c person-odb.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -I../../install/include -Os -c -DDATABASE_SQLITE driver.cxx
   arm-bcm2708hardfp-linux-gnueabi-g++ -L../../install/lib -o driver driver.o person-odb.o -lodb-sqlite -lodb -lsqlite3

The result of the last command is the driver executable. To run it on the target, besides the executable itself, we will also need to copy over the shared libraries from the install/lib sub-directory. The libraries that we will need are: libsqlite3.so.*, libodb-X.Y.so, and libodb-sqlite-X.Y.so. On the target we will also need to make sure that the dynamic linker can find them. While how exactly to achieve this depends on the target, installing them into /usr/local/lib or adding their directory to the LD_LIBRARY_PATH environment variable will work for Raspberry Pi or any other GNU/Linux derivative. For example, assuming that our executable and all the libraries are in the current directory (on the target), we can run the example like this: 

   raspberrypi $ uname -a
   Linux raspberrypi 3.2.27+ #3 PREEMPT Sat Dec 15 18:52:34 SAST 2012 armv6l GNU/Linux
   
   raspberrypi $ export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH
   raspberrypi $ ./driver --database /tmp/test.db
   Hello, John Doe!
   Hello, Jane Doe!
   
   count  : 3
   min age: 31
   max age: 33
[edit]

Minimizing Runtime and Generated Code Size

To give an indication of the footprint one can expect when using ODB on mobile/embedded systems, the stripped hello example executable when built with static SQLite and ODB runtime libraries is 533Kb. While this may seem like a lot for such a simple application, keep in mind that it includes the whole SQLite database as well as the libodb and libodb-sqlite runtimes, all of which are "once-off costs", that is, they don't change with the number of persistent classes used by the application.

A more useful size breakdown can be obtained from the build that uses shared libraries (again, all stripped): 

   35348  driver
   38244  libodb-2.1.so
   107532 libodb-sqlite-2.1.so
   504232 libsqlite3.so.0.8.6

As you can see, the driver itself, which contains the generated code for one persistent class and one view, is only 34Kb. Note also that the combined size of all the libraries and the executable (669Kb) is about 25% greater than that of the static executable.

The ODB compiler implements fine-grained control over various features provided by the generated code. By not enabling functionality that is not needed by your application you can greatly reduce the generated code size and the resulting application footprint. In particular, the following ODB command line flags control optional functionality provided by the generated code: 

   --generate-query
   --generate-prepared
   --omit-unprepared
   --generate-session
   --generate-schema

Also, when generating database schema embedded into the C++ code, it can be wasteful to keep that code in the main executable (and thus in the device's memory) all the time. For example, if you need to create the database schema only once when the device is first turned on, then it may make sense to factor the schema creation function into a separate executable. This can be achieved with the --schema-format separate option which instructs the ODB compiler to generate the schema creation C++ code into a separate source file.

If your application is single-threaded, then it is also possible to slightly reduce the ODB runtime sizes by disabling multi-threading support. To achieve this, pass the --disable-threads option to the configure commands.

To reduce runtime memory usage and dynamic memory allocations in persistent classes, consider using char[N] arrays (or C++11 std::array<char,N>) instead of std::string or similar for representing string and binary data. For example: 

   #pragma db object
   class person
   {
     ...
     
     char first[32];
     char last[32];
     
     #pragma db type("BLOB")
     char public_key[1024];
   };
Retrieved from "http://wiki.codesynthesis.com/Using_ODB_on_Mobile_and_Embedded_Systems"
Personal tools