ESE C#/C++ toolkit 1.2
for Microsoft Extensible Storage Engine
If you want us to get in touch or have a question
 

Introduction

Many developers are unaware that one of the most high performance, innovative and rich database engines available is built in as standard to all copies of Microsoft Windows. Microsoft have recently published the APIs and it is now available to use for free.

Extensible Storage Engine (ESE) is part of a special subset of Windows technologies known as the Trusted Computing Base (TCB). The technology is used as the primary database in such notable applications as Windows Active Directory, Microsoft Exchange, and Windows Vista.

The database technology offers very high performance, scalability to 16 terabytes of data per database file, enterprise features such as hot backup and a high degree of flexibility supporting for example transactions nested up to 7 levels.

ESE can be used to add lightweight data storage features to Windows applications and is also provides a very attractive foundation for custom database server applications where peformance is especially important.

Numerous innovative features supported by the database include:

  • Tagged columns
    A tagged column can store multiple values. For example, you could store all the tags associated with a row of data (such as a forum post) in a single row. This denormalization can massively increase performance when the column is indexed, as it effectively removes joins from lookup operations.
  • Reference counted long columns
    Long column values can be copied between rows in such a way that no data is actually copied, and instead the underlying data becomes shared and reference counted. This makes it possible to add a large video file, say, to multiple rows efficiently, again eliminating join requirements.
  • Conditional columns
    A conditional column controls whether a row appears within specified indexes. This makes it possible to remove and re-surface rows in indexes on demand - for example to implement temporary delete or archival features, or as the basis for high performance garbage collection strategies.
  • Version and Escrow columns
    A version column is incremented each time its row is modified, and escrow columns contain counts that may be safely updated outside of a transaction to improve performance (rather like interlocked memory values).
  • See Microsoft's low-level API reference to find out more.

Building with ESE

The ESE engine is encapsulated within a Windows system library, and its APIs may be accessed from C++ by linking to the ese.h Windows SDK header file. Using the standard API, the engine is directly linked into your application (which means your application gains exclusive control over the database files it manages).

The raw ESE API is complex and contains almost 150 separate functions. These functions in turn often need numerous parameters, which themselves may comprise complex structures.

Using ESE in out-of-the-box form involves the following considerations:

  • Writing code to mask the low-level complexities of the API (necessary for productivity and robustness).
  • Using the APIs to instruct the engine to build tables, perform highly optimized lookups and joins and so on as per the requirement.
  • Designing effective interoperability between managed and unmanaged code where, for example, you wish to write the bulk of application in C# or Java.

Our library and toolkit helps you address these issues.


Our library

We have developed a library of C++ and C# classes and provide an extensive sample application written in C# to assist those developers wishing to leverage Microsoft Extensible Storage Engine (the sample application comprises production code in use today). It has the following components:

  • C++ classes that provide a higher level abstraction of Extensible Storage Engine e.g DbCursor, DbTable, DbSchema etc. These well designed abstractions allow you to seamlessly work with transactions and other database considerations inside object-oriented C++ code.
  • C++ classes that allow database query operations to be created simply by instantiating C++ templates with parameters. For example, to select rows based upon multiple index criteria and iterate through the result set.
  • C# classes and functions that allow for the integration of database functionality residing inside bespoke C++ "database driver" dlls into managed code. For example to allow C# code to call into a C++ function performing a select and receive the results via safe but high performance interop.
  • An example forum and blog technology built using Javascript, C#, Web services and C++/ESE. This uses some of the advanced features of ESE and illustrates performance gains of thousands of percent over traditional RDBMS technology.

At the current time, samples from the C++ and C# classes are offered from our library for free. These are supplied under the The BSD License.

Free C++ and C# Downloads

We have made a packages of files available as a free download, which can assist with new ESE projects. Please come back and visit often as we plan to make an increasing number of files available, and hopefully eventually start an open source development effort. They are licensed using The BSD License below.

Please contact us if you are interested in licensing the full library, toolkit and sample application.

open_src_release.zip


The BSD License

Copyright (c) 2008-2009, Centre Technologies Ltd.
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • Neither the name of Centre Technologies Ltd, nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.