Extending OpenXDK

Adding new kernel functions

The xboxkrnl.exe provides over 350 functions/variables that can be used. Each of these functions has to have the signature defined correctly - which is a time-consuming process and is typically only done for each function as it is needed. Fortunately, most of the signatures for the XBOX kernel functions match (or are very similar to) their NT/2000 equivalents.

Update C and H files

You can easily see which functions have not yet been defined by looked at the files in the include/xboxkrnl directory. For example, you might see a function in ex.h that is declared like:

XBSYSAPI VOID *ExAllocatePool;

Likewise, in the src/hal/xboxkrnl/ex.c file, you will see something like:

XBSYSAPI VOID *ExAllocatePool = 0;

To make this available for use, you need to figure out what parameters it takes. Looking at the equivalent NT/2000 function signature often gives you enough information. Alternatively, you may be able to ask someone who has an official Microsoft XDK to tell you. In this case, there is only one parameter - an unsigned long indicating how many bytes is requested. Knowing this, you should modify the include and src files. The ex.h will look something like:

XBSYSAPI EXPORTNUM(14) PVOID NTAPI ExAllocatePool
(
  IN ULONG NumberOfBytes
);

The newly implemented ex.c will look like:

XBSYSAPI EXPORTNUM(14) PVOID NTAPI ExAllocatePool
(
  IN ULONG NumberOfBytes
)
{
  return NULL;
}

Note that ex.c actually contains a function body that looks like it returns something. Don't worry... this code never gets called. These are just stubs so that we can create an xboxkrnl.lib that gets linked against. At run time, the code jumps to an external function (identified by ordinal 14 in this case) in a file called xboxkrnl.exe. This functionality is actually provided by the XBOX kernel (not by our dodgy stubs).

Update xboxkrnl.exe.def

Now, there is one more step before this function is available. You need to update the function signature in the xboxkrnl.exe.def file so that it matches what the linker is expecting. For example, the line that would be in the xboxkrnl.exe.def file would originally look like:

  ExAllocatePool            @  14 NONAME

You need to change it to:

  ExAllocatePool@4          @  14 NONAME

The @4 indicates the total length of all parameters. You can calculate this yourself manually, or you can do what I do and compile a very small OpenXDK program that links to this function:

#include <openxdk/openxdk.h>
void XBOXStartup()
{
  ExAllocatePool(0);
}

and then run PEDUMP against the .EXE file. You will see something like:

Imports Table: 
  xboxkrnl.exe 
  OrigFirstThunk:  0000ECCC (Unbound IAT) 
  TimeDateStamp:   00000000 -> Thu Jan 01 11:00:00 1970 
  ForwarderChain:  00000000 
  First thunk RVA: 0000ED10 
  Ordn  Name 
     4  ExAllocatePool@4 
    49 
   156 
   184

Use this as a guide to determining the length of the parameters. Update the xboxkrnl.exe.def file, remake your xboxkrnl.exe, rebuild your example program, and you should see something like:

Imports Table: 
  xboxkrnl.exe 
  OrigFirstThunk:  0000ECCC (Unbound IAT) 
  TimeDateStamp:   00000000 -> Thu Jan 01 11:00:00 1970 
  ForwarderChain:  00000000 
  First thunk RVA: 0000ED10 
  Ordn  Name 
    14
    49 
   156 
   184

Complex data types

You may have noticed the convention of using EXPORTNUM(xx). This macro doesn't actually do anything... it is just a visual indicator as to the ordinal that the function is meant to be defined as. Likewise for the IN and OUT modifiers; they are just indicators for when you are browsing the source code.

There are some functions (for example, NtCreateFile) that take complex data types such as POBJECT_ATTRIBUTES. Definitions for most of these structure are usually defined in <windows.h>, however, we do not want to use those definitions for a couple of reasons:

Including <windows.h> introduces a bunch of Cygwin definitions that we don't want
Often the elements of the struct definitions are slightly different for the XBOX.

As a consequence, there is a file called include/types.h that contains all the Windows specific structures, macros and data types. If you add new data types, this include file is the place to add them.

Import Address Table

Hopefully, you will never need this information, but it caused me a lot of pain until I figured out how to patch CXBE to work around it.

One of the data structures in a PE file is what is known as an Import Address Table (IAT). This table is used by the XBOX kernel (in fact, also the NT/2000 loader) to determine what addresses various functions are so that it can set up various jump tables. The IAT contains a collection of thunk addresses, but the one we care about most is the first thunk address - which is the entry point for the application. When we run CXBE to convert the .EXE to a .XBE file, it finds the address of the IAT from the DataDirectory, gets the first thunk address and sets that as the entry point in the XBE header.

However, unfortunately, GCC doesn't set the address of the IAT in the DataDirectory. As a consequence, CXBE was unable to find the entry point of the application. I have provided a patch to the source for CXBE that handles these cases, and I ship a binary version of CXBE with the OpenXDK distribution. The point of this, though, is that if you do not have the updated version of CXBE, OpenXDK applications will not run.

newlib

newlib is an awesome collection of code to provide an alternate implementation of libc. However, it caused me an inordinate amount of pain when porting to the XBOX. Because our target platform is essentially i386, Cygwin thinks that it is going to be around and tries to help with providing its own implementations of various things (malloc, in particular, was a major pain). In an attempt to get around this, I created a new target: i386-pc-xbox. Because Cygwin now thinks it is cross-compiling, it provides a little less "help". However, a side-effect of this is that automake/autoconf now also thinks we are cross-compiling, so it wants to use cross-compiler style names. This is why you need to create the symbolic links describe in the Installation Guide.

I initially tried to get the libgloss working, but couldn't figure it out properly. In the end, I finished up putting everything I needed in

/newlib-1.12.0/newlib/libc/sys/xbox/syscalls.c

I am working on fixing up any references in the configure scripts to libgloss, but some may still remain. The important thing to remember, is that although we produce a number of files (crt0.o, libc.a, libg.a, libm.a, libnosys.a), the libgloss (libnosys.a) library is not used. Hopefully, I will figure out how to stop automake from creating it.

The command line that I use to configure newlib is:

./configure --target=i386-pc-xbox --prefix=/usr/local/openxdk --with-newlib --without-headers

There a couple of things that I would like to add to the newlib implementation. Specifically:

Implement the execve() function. I have seen code that can launch an XBE, but haven't actually done it myself yet
Implement the fork() function. Likewise, I have seen code that calls the PsCreateSystemThreadEx() function to do this.
Fix the implementation of times(). It currently returns the time since the system was booted, not since 1970. Oops!

SDL

libsdl is a sensational multimedia library that really is designed to port to other platforms with a minimum of ease. Yet again though, I was defeated by automake/autoconf, so wrote my own makefile to compile SDL for the XBOX. It is really very simple! Just compile like this:

make -f makefile.xbox

Currently supported are video, events, timer and joystick. The video driver currently always renders 640x480 pixels. If you request another resolution, my implementation centres it within the 640x480 view. One of these days, I will figure out how to change the video modes... Next on the hit list is definitely audio support.

Missing Features

This section describes general pieces of functionality that are currently not present. My general philosophy is to port, rather than write from scratch. Typically, I try to port existing software from Linux/Windows and then if there is not an already fairly well accepted standard, provide a HAL api to make it easy for developers to use (Joystick support is an example). However, if there is a well entrenched way of using a particular piece of functionality, I try not to re-invent the wheel (for example, for networking, I hope to be able to provide typical socket-style calls - socket(), ioctl(), etc).

Back to Home Page