Play with VMFlexArray

I explained VMFlexArray in my last post, and went to some effort to get it approved as a topic for this years Google Summer of Code, kindly under the umbrella of the GNU Project, Classpath, and IcedTea.

Judging from the fact that only one Google Summer of Code student proposed to work on VMFlexArray, it seems as though the concepts might be difficult for students to conceptualize.

For that reason, I built a small VMWare image that anyone can freely download to experiment with VMFlexArray.

In the process of doing so, I also converted FB4J to use JNA instead of JNI, and I'm very glad I did. The average refresh rate has now gone up by a significant factor to above 40 fps, and I haven't even begun to optimize anything.

The way I would suggest someone begin experimenting with VMFlexArray is as follows:

1. Download and uncompress the VMWare image (there is a .vmdk inside the .vmwarevm folder for those who do not use Mac OS X / VMWare Fusion)
2. Run the VMFlexArrayLinux virtual machine
3. Log in with user 'root' and an empty password
4. Run 'ifconfig eth0' and write down the IP address
5. Open a terminal session on the host OS and ssh into the IP address from above (i.e. ssh root@[ip_addr])
6. run the demo ./fb4jdemo

You should see a white background and some colourful balls bouncing around on the VMWare's virtual SVGA II device.

I hope you're curious enough to begin to dissect the demo being run.

After some investigation, it should become obvious that JamVM and Classpath are being used under the hood for java. One might also notice that there is a java compiler (javac) in the image courtesy of Eclipse Compiler for Java (ecj). Some clever people will probably also notice /var/db/pkg.sqfs, which is a SquashFS version Gentoo's database of installed packages (tying all installed free software back to source repositories).

Then, please take note that there is a portage overlay located at /usr/local/portage/java_overlay, as well as a patch located in /usr/src. The portage overlay represents changes required to demonstrate VMFlexArray using jamvm and gnu classpath. The patch in /usr/src represents the one patch required to perform double-buffering using the VMWare frame buffer.

I went to some length to ensure that my patch set was easy to reproduce. Therefore, all required changes to JamVM are located in my feature/vmflexarray-demo branch, and all required Classpath changes are located in my feature/vmflexarray-demo branch. Also, all required changes to JNA-Posix are in my branch feature/ioctl.

My updated FB4J changes are currently in the feature/jna branch. These changes mean that FB4J does not require any of its own native components. They will be merged into master eventually, but I just need to a) clean up the README, and b) possibly machine-generate enums and constants from /usr/src/linux/fb.h

This is where it gets fun.

The next task in understanding VMFlexArray, for anyone who is curious, is to take the Fb4jDemo code and modify it to draw a different video effect. The good news is that there is a built-in java compiler (ecj) in the VMWare image, so that's rather easy.

Next, do something with FB4J that I haven't done yet - use it to read a Video 4 Linux device ;-) This will require you to roll your own kernel and add UVC, V4L, and other modules in.

Here's a screenshot of my latest run. Don't forget to download the compressed VMWare image to experiment with from here.

Happy frame buffer drawing!

In the mean time, I will continue to experiment on my end. Once I have some free time, I might attempt an OpenJDK port of the VMFlexArray changes.

Please note that the software is still very beta - e.g. I am currently getting an undiagnosed segfault after some time. I believe I ran into this problem before I did a major cleanup of the code, and will review it when I have some time.

MappedByteBuffer.hurray()!: Programming the Linux Framebuffer in Java & VMFlexArray Explained

I recently travelled to Belgium to participate in FOSDEM. This year, I gave two presentations:

• MappedByteBuffer.hurray()!: Programming the Linux FrameBuffer in Java & VMFlexArray Explained. See here.
• Internet of #allthethings: Using GNURadio Companion to Interact with an IEEE 802.15.4 Network. See here.

The topic of this post will focus on the former. Specifically, VMFlexArray.

Currently, there is no Java Virtual Machine in existence that allows a developer to reference off-heap memory regions as Java arrays - e.g. via byte[] or int[]. That is, all java arrays that the VM deals with must be contiguously allocated at instantiation time.

What I mean by that, is that when the VM instantiates an integer array object, e.g. int[] x = new int[ length ], it typically allocates memory (now careful, I'm going to use some C teriminology here) for an object struct (2 uintptr_t in JamVM) which represents the instance of the int[] object, followed by 1 uintptr_t, which represents the length of the int[] object, followed by exactly length uintptr_t items (on a 32-bit machine) or length / 2 uintptr_t items (on a 64-bit machine) to represent the data.

VMFlexArray

VMFlexArrays are slightly different. For the same case as above, where a new int[] is allocated on the Java heap, the VM would allocate an object struct (2 uintptr_t in JamVM) which represents the instance of the int[] object, followed by 1 uintptr_t to represent the length of the int[] object, followed by 1 uintptr_t to point to the int[] data, followed by the data itself.

What makes VMFlexArrays different, and what makes them flexible (and arguably way better than what most JVMs use today) is that they include that extra uintptr_t to point to the data which could exist anywhere in virtual memory. That means, obviously, VMFlexArrays can point to contiguous data that the JVM would allocate for a regular array, but it also means that it can point to an arbitrary location - and still cooperate with the garbage collector. Indeed, the object lifecycle remains unchanged for VMFlexArrays if the garbage collector avoids releasing memory regions with free(3) if the VMFlexArray pointer does not point to the next contiguous memory address.

VMFlexArray is a solution I came up with that allows one integrate off-heap memory regions into the Java Virtual Machine - e.g. a native external thread that allocates memory using malloc(3), or pages mapped from a device such as /dev/video0 using mmap(2).

Buffer Views

Perhaps the aspect of VMFlexArrays that I found most useful, that I somehow forgot to mention during my talk, is that they rather trivially allow the following code snippet to work as expected. Specifically, an IntBuffer derived from a ByteBuffer with a backing array should be able to provide an int[] backing array view of the same virtual memory.

Currently this code, which should work pretty seamlessly, fails miserably.

ByteBuffers allow themselves to be viewed as IntBuffers or LongBuffers or ShortBuffers. Pretty brilliant! Well... if it worked it would be brilliant. The fact is, as shown above, one cannot wrap a byte[] into a ByteBuffer, view it as an IntBuffer, and then call IntBuffer.array() to get an int[] view of the original byte[]. That would make the NIO API complete, in my opinion, and this feature is sadly lacking.

With VMFlexArrays, that problem is solved.

I've even used this code to memory map the Linux FrameBuffer and animate a bunch of bouncing balls :-) It works quite well.

There's even a massive speedup associated with access to the underlying byte[] from a ByteBuffer and even more so viewing the ByteBuffer as an IntBuffer, with access to the underlying int[].

I am definitely interested in enabling these changes to make it into OpenJDK, and I feel that the community at large would benefit greatly from them. As my time is rather limited these days, I might prefer to mentor a student to make these changes in the Google Summer of Code, 2015, if OpenJDK was a mentoring organization. Otherwise, I would be open to mentoring a student under the umbrella of JamVM or GNU Classpath as a mentoring organization.

A Distinctive Lack of Simple Complexity in Java

Simplicity is a wonderful thing. When problems become increasingly complex, sometimes a simple action, like going for a run or turning off the radio, can often simplify the solution.

Java generally makes programming tasks simple; there is "no need to worry" about memory allocation and garbage collection, and it "just works" on basically any platform. However, unlike its (arguably) simpler counterpart C, Java does not support complex numbers in the standard runtime. There are some independent efforts to create a generic Complex<T> class, which would be necessary for completeness in Java's Object framework, but the primitive complex numeric types are noticeably absent. As far as I'm aware, it's also impossible to define a new native type in Java without modifying the JRE / JDK directly. Why Java didn't add complex numeric types from the beginning is a mystery to me. It might have something to do with Windows compatibility, but I can't say that for certain.

Who uses complex numbers in software anyway?

Although it's clearly simple in any language to define a set of functions (methods) and structures (classes) for dealing with complex numbers, if the API is defined at the language level, then all users of that language can share the advantage of a common, complete, and compatible library for their code. Complex numbers are used heavily by engineers and mathematicians. Generally any mathematical tool (such as Matlab, Mathematica, or GNU Octave) requires complex number support for completion. Complex number support is also heavily used by more specific electrical, RF, signal, and mechanical engineering software - essentially anything that deals with a time-variant oscillatory system. Mobile phones do complex math in the baseband processor, so one could say that we ALL benefit from standardized complex number support.

Personally, I have been hacking with the USRP2 (Universal Software Radio Peripheral) from Ettus Research, Inc, and have been using a custom Java / JNI library to interface with it. Since the data that the USRP2 receives and transmits is in the baseband domain, it is generally always complex. The C / JNI portion of my code can handle complex numbers natively, but when I use my Java interface, my options were limited. I either needed to access the data real and imaginary data independently as float or short variables and arrays, or I needed to write a simple Java class for complex numbers.

Why does native complex number support in Java matter?

Consider the overhead of using a single java.lang.Integer object. First of all, every Integer class must inherit from java.lang.Number, but also from java.lang.Object. Static (class) methods and non-static (instance) generally do not introduce much overhead, since they only exist (maximally) once per Class and share everything but their instance information with other instances. However, for each Integer instance, there exists a non-trivial number of fields, including but not limited to i) a hash code, ii) a reference to the Class, iii) synchronization primitives for wait(), notify(), etc, iv) various flags indicating things like finalization, constant status, etc. There are probably also some additional instance fields inherited from Number.

Now, consider the simplest of Integer collections: an Integer[] (array of Integer objects). Aside from the (generally) 4-byte value of the Integer itself, there are undoubtedly at least an order of magnitude more bytes just for instance information of the Integer object (hash, references, synchronization primitives, flags, etc). Lets assume that there are 28-bytes of additional instance information. For an array of 512 Integer objects, the approximate storage requirement is (28 + 4) * 512 = 16384 bytes, which dramatically exceeds the (approximate) 2048 bytes that a simple int[512] would require.

In order to support a complex Integer type, for the same array length, there would be approximately 32768 bytes versus the 4096 bytes that would be required in C. In actual JRE implementations, there may be some optimizations for arrays of Numeric objects, but when all is said and done, each element in such an array must still be a fully qualified Object upon access.

In C (GNU C at least), any numeric data type can be augmented with the complex keyword, and I think that Java would benefit greatly from introducing the complex keyword into the language specification as well. Naturally, there would need to be special consideration made for various things such as division, casting, etc, but overall it should not be very difficult.

Posix and GNU C take a very simple and elegant approach to complex numbers. They define a special variable, I (and _Complex_I), as well as a set of standard math.h extensions. However, Posix C only defines functions for dealing with double and float complex numbers (much like the majority of functions in math.h). In GNU C, one can additionally use any numeric type with complex numbers. The storage space doubles for a numeric type that is declared complex, with the real and then the complex portion being allocated contiguously. So if I define a complex int x, then & ( __real__ x ) == & x, and & ( __imag__ x ) == & x + 4, while sizeof( x ) == 8. The C language takes this allocation into account with arrays and pointer indexing as well, so & x[5] == & x[4] + 8, and so on.

Although I really have no time to work on this personally, at some point if I do have time, I would probably modify JamVM and the GNU Classpath to implement natively supported complex numbers. With a simple open-source implementation as a reference, putting together a JSR for official adoption would probably widely accepted and welcomed by the majority of the Java community.

If anyone is aware of an existing JSR for the complex keyword and associated modifications, classes, etc. Please let me know. Otherwise, if anyone is interested in collaborating on this, please leave a comment below.

Likewise, I'm fairly certain that C# does not support native complex numbers, although Python might. So, if anyone would like to hack Mono and get native complex types built-in, please leave a comment below.

VAI Prime Overlays Accessible Via Layman

After making my vuze-bin overlay accessible via layman, I decided to apply the same approach to my ts72xx related overlays on VAI Prime.

You can see the overlays via layman with

layman -o http://vaiprime.visibleassets.com/~cfriedt/layman.conf -i [overlay-name]

So far, the available overlay-names are as follows:

• jvmcp (Miniaturized Java environment suitable for the ts72xx boards)
  
  • custom JamVM (1.5.1)
  • custom GNU Classpath (0.97.1)
• vai (mainly custom apps for the ts72xx)
• custom JamVM (1.5.1)
• custom GNU Classpath (0.97.1)
• custom battery monitor
• maverick (overlays necessary for creating a EABI/MaverickCrunch toolchain for the ts72xx)
• patched binutils
• patched gcc
• patched crossdev
• patched uclibc
• ts72xx (overlay of applications for the ts72xx)
• patched fastjar
• patched busybox
• patched screen
• patched portage-utils
• patched bash
• patched java-config
• patched jamvm
• patched rxtx
• patched ant-core
• patched ant-eclipse-ecj
• patched eclipse-ecj
• patched gnu-classpath
• patched mpfr
• patched nfs-utils
• patched libnfsidmap
• patched openvpn
• patched dropbear
• patched wpa_supplicant
• patched binutils (2.15)
• patched gcc (3.4.6
• patched db
• patched psmisc
• ts72xxtc (binary-compatible toolchain for factory TS filesystem)
• patched glibc
• patched linux-headers

It would probably be wise to move at least the binutils and gcc from ts72xx to ts27xxtc.

Anyway, check out the overlay with

layman -o http://vaiprime.visibleassets.com/~cfriedt/layman.conf

Having Fun with Embedded Gentoo on the TS72xx SBC

Hi everyone!

Having left Kiel last week and after a week of being back in Montréal doing some hard work, I've gotten my toolchains, overlays and binary package repositories for the TS72xx board to a very stable state.

I'm now using dropbear, openvpn, jamvm-1.5.1, classpath-0.97.1, and qmerge, thanks to the ease of building binary packages using Gentoo/Portage, xmerge, etc. Here are a couple of screenshots that can hopefully do justice to all of the work I've done with the 'compatibility' toolchain. The compatibility toolchain and binaries have been tested to be fully compatible with the kernel, libc, and so on distributed by Technologic Systems.








The toolchains, overlays, and repositories are for both arm-unknown-linux-gnu and armv4tl-maverick-linux-gnueabi. I've also updated the documentation on the Gentoo-Wiki:

Gentoo for the TS72xx (Old Toolchain)
Gentoo for the TS72xx Single Board Computer (Full Distro)

I had to fix the __NR_waitpid definition (removed it) in the linux kernel headers which solved a lot of glibc related issues I was having.

Next, I compiled busybox, etc, etc, got qmerge working on the board, and I've also compiled jamvm-1.5.1 and the classpath-0.97.1 . There was a slight problem with cross-compiling the GNU classpath shown below in the classpath bug.

One really strange problem that I had was with portage-utils. The md5sum code comes directly out of the busybox source code. The md5sum comman works perfectly on my board, but as is, the md5 calculations that qmerge made were always wrong. I didn't really want to spend too much time fixing the source code, so instead, I just created a patch for qmerge so that it calls 'md5sum' through popen. That fixed the problem, although the overhead is obviously very high. I should probably put in a patch since Ned Ludd, the original portage-utils author has been so helpful to me on the gentoo-embedded list. Thanks again Ned ;-)

I also noticed that busybox wouldn't reboot the system (using the older toolchain) so I had to make a patch which used the busybox-1.0.0 code for 'reboot'.

I'm hoping that some autotools / libtool genius decides to fix the classpath bug below, because I really don't want to spend any more time on it than I have to. The autotools should really _not_ abandon supporting cross-toolchains. There should really be a regular check made whenever AC_CHECK_LIB or AC_SEARCH_LIBS are made to see if the library location pulled from the .la file actually reflects the location of the .so or .a library in question.

As always, see my overlays for all patches to the original source code.

Some bugs filed:
Gentoo Bug #213690
Classpath Bug #35684

My experiences with Qemu - Part II

So I've managed to get this far with Qemu:

(qemu) Uncompressing Linux......................................................... done, booting the kernel.
Linux version 2.6.20.6 (cfriedt@sith) (gcc version 3.4.4) #35 Thu May 31 21:57:50 CEST 2007
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00003137
Machine: ARM-Versatile PB
Memory policy: ECC disabled, Data cache writeback
CPU0: D VIVT write-through cache
CPU0: I cache: 4096 bytes, associativity 4, 32 byte lines, 32 sets
CPU0: D cache: 65536 bytes, associativity 4, 32 byte lines, 512 sets
Built 1 zonelists. Total pages: 65024
Kernel command line: root=/dev/nfs nfsroot=192.168.7.1:/usr/gentoo_root,uid=0,gid=0,rsize=32768,wsize=32768,timeo=14,nfsvers=3,rw rw ip=192.168.7.2:192.168.7.1:192.168.7.1:255.255.255.0:qemu console=ttyAMA0,115200
PID hash table entries: 1024 (order: 10, 4096 bytes)
Console: colour dummy device 80x30
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 256MB = 256MB total
Memory: 258048KB available (1516K code, 174K data, 100K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
NET: Registered protocol family 16
PCI core found (slot 11)
PCI: bus0: Fast back to back transfers disabled
PCI map irq: slot 0, pin 1, devslot 12, irq: 27
NET: Registered protocol family 2
IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
TCP established hash table entries: 8192 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 8192 bind 4096)
TCP reno registered
NetWinder Floating Point Emulator V0.97 (double precision)
CLCD: unknown LCD panel ID 0x00001000, using VGA
CLCD: Versatile hardware, VGA display
Clock CLCDCLK: setting VCO reg params: S=1 R=99 V=98
Console: switching to colour frame buffer device 80x30
Serial: AMBA PL011 UART driver
dev:f1: ttyAMA0 at MMIO 0x101f1000 (irq = 12) is a AMBA/PL011
dev:f2: ttyAMA1 at MMIO 0x101f2000 (irq = 13) is a AMBA/PL011
dev:f3: ttyAMA2 at MMIO 0x101f3000 (irq = 14) is a AMBA/PL011
fpga:09: ttyAMA3 at MMIO 0x10009000 (irq = 38) is a AMBA/PL011
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky
smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre
eth0: SMC91C11xFD (rev 1) at d080a000 IRQ 25 [nowait]
eth0: Ethernet addr: 52:54:00:12:34:56
mice: PS/2 mouse device common for all mice
TCP cubic registered
ieee80211: 802.11 data/management/control stack, git-1.1.13
ieee80211: Copyright (C) 2004-2005 Intel Corporation
input: AT Raw Set 2 keyboard as /class/input/input0
eth0: link up
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
IP-Config: Complete:
device=eth0, addr=192.168.7.2, mask=255.255.255.0, gw=192.168.7.1,
host=qemu, domain=, nis-domain=(none),
bootserver=192.168.7.1, rootserver=192.168.7.1, rootpath=
Root-NFS: unknown option: uid=0
Looking up port of RPC 100003/2 on 192.168.7.1
Looking up port of RPC 100005/1 on 192.168.7.1
VFS: Mounted root (nfs filesystem).
Freeing init memory: 100K
init: must be superuser.
Kernel panic - not syncing: Attempted to kill init!

It's looking fairly bright now ;-) All I have to do is figure out why (this has absolutely no real reason) Linux would start up as someone who isn't the superuser
... hmmf..

Weeee !!! I just added an init= option to the 'append' section of Qemu!!! sahweet!!! She's ready to roll ;-)

The option to enable 16-bit UID system calls must be selected as well, otherwise root (0) will get mapped to 0xffffffff (-1 in 2's compliment == -EPERM).

My experiences with Qemu - part I ... was "Building GNU Classpath for the ARM (Cont'd)"

Ok, so it looks like the external USB hard disk idea was not as good as I was hoping. The problem is that the board just has too little RAM !!! (to compile such a package as the GNU Classpath)

What happens in either the NFS case or the USB HD case, is that eventually the swapd and the jikes process enter this vicious cycle, where jikes caches the first half of what it needs to compile, swaps it out, and then caches the second half of what it needs to compile - clearly if it doesn't have 100% of the things it needs to compile in RAM (this occurred even without using -pipe) then it will enter an infinite loop (it would be nice if that was detected by jikes - i think that's CRC32 or something?).

In any event, with the suggestion that my boss Dave gave me, I'm now working on emulating an appropriate kernel, and using the appropriate uclibc-softfloat userspace for the ARM, using Qemu. This should solve all of my compilation woes because I would then be able to have enough actual physical RAM to compile everything natively. I believe that the VersatilePB platform available in Qemu has a limit of 256MB RAM though. That should be enough (I assume).

Aside from the target having a different name, the main other differences between the TS-7xxx boards and the Versatile PB are:

• The Versatile PB uses an ARM926T chip instead of an ARM920T
• The Versatile PB supports an SMC 91c111 ethernet device instead of the ep93xx device.
• The Versatile PB names its serial devices /dev/ttyAMA[0-n] instead of /dev/ttyAM[0-n]

All of the exact details are here:

http://fabrice.bellard.free.fr/qemu/qemu-doc.html#SEC53

Now, a brief overview of the differences and what they will mean.

1. The ARM926T chip vs the ARM920T chip. To quote from the linux kernel help page:
   

   This is a variant of the ARM920. It has slightly different instruction sequences for cache and TLB operations. Curiously, there is no documentation on it at the ARM corporate website [EDIT: There is now].
   
   Say Y if you want support for the ARM926T processor. Otherwise, say N.

   
   I tried compiling for the Versatile PB without ARM926T support and that didn't even compile - it would be nice to have the ARM920T as an option here too. Anyways, that's not really a problem. As long as I can run Qemu with an ARM chip emulated and use an NFS root to compile, that's all that matters.
   
   
2. eth0 is eth0 ... I doubt it matters what hardware is underlying for my purposes.
   
   
3. Some scripts and applications will have to be aware of the difference of platform, if i end up coding scripts and things of the like on the emulator. I just as soon wouldn't, but it could be quite beneficial. In any event, that's not a real issue, it's as simple as replacing a string from my perspective. The cool part about that is that we could end up simulating our hardware, including the EM noise model - cool ;-)
   

I'll keep you posted!

Building GNU Classpath for the ARM (Cont'd ...)

I spontaneously decided to copy over the entire Gentoo stage3 filesystem to my external hard-disk, which is recognized as a general USB storage device under linux. NFS was taking waaaaay too long for this compilation. I had checked everything out with top, under linux, played with the swappiness parameter in /proc/sys/vm, and also attempted to renice the jikes processes' priority to a whopping 17! Non of those had a major effect on the cpu-usage of the jikes process. The highest that I'd seen it was 10% for a whole 2 seconds.

The problem lies with NFS root and the overhead associate not only with RPC, but also the fact that my swap was also on the NFS root, effectively squaring the complexity of the overhead. With my Gentoo stage 3 root filesystem located on a locally attached usb storage device, along with the swap file, the speed of compilation increased dramatically! The jikes process has just started and its already achieved 50% of the cpu time! That's up from an average of maybe 3% with the tweaking I mentioned previously.

I'd better mention this to Dave as well, so that he's not stuck ever repeating my stupid mistake of compiling on an NFS root!

Building the GNU Classpath for ARM

My preferred method now for building packages on the TS-7xxx series of single board computers (SBC's) is to use the gentoo stage3 filesystem as an NFS root, which comes with a C/C++ compiler and most of the necessary utilities. It is much slower, but this way I don't have to work around any strange cross-compiling issues or annoying pkgconfig native-executable requirements which tend to pollute configure scripts for most higher level applications.

The target I'm building for is an ARM 920T processor from cirrus - the EP93xx series of processors - which have only recently acquired a decent level of support in the Linux kernel. Also, as of gcc-4.1.1 the Thumb instruction set is supported as well, although I don't really use it. The ARM EABI is quite useful for those that require floating point functionality, but most of what my company does requires integer math for the most part. For that reason, the stage3 filesystem I use for my NFS root is the arm-softfloat-linux-uclibc stage3 filesystem located here:

http://adelie.polymtl.ca/experimental/arm/embedded/stages/stage3-arm-uclibc-softfloat-20050811.tar.bz2

However, please feel free to select any Gentoo mirror from this site:

http://www.gentoo.org/main/en/mirrors.xml

I've posted several sets of instructions for those that would also like to use the 2.6 kernel I've compiled for the TS-7xxx boards as well. If you're interested, please see the mailing list archives:

http://tech.groups.yahoo.com/group/ts-7000/message/6574

You can also download the kernel directly from here:

http://vaiprime.visibleassets.com/~cfriedt/zImage-2.6

and the config, here:

http://vaiprime.visibleassets.com/~cfriedt/linux-2.6.20.6.config

I recently changed my kernel configuration to include swap functionality, specifically because compiling the gnu classpath for the arm used over 3 times as much memory than was available on the board.

The tricky part though, is that Linux doesn't play very well with swap files on an NFS filesystem. But after doing a bit of googling, I found that there is 1 commonly used work-around.

# dd if=/dev/zero of=/mnt/swapfile bs=1024 count=$((1024*256))
# losetup /dev/loop0 /mnt/swapfile
# mkswap /dev/loop0
# swapon /dev/loop0

In the above section, I've created a 256MB swap file for the linux kernel, and it is accessed by the kernel through a loop device, which offers some apparently greater level of control than simply using swapon /mnt/swapfile.

I'm still compiling the gnu classpath, and I'm not sure exactly how far along the process is, but here are some of the memory usage figures:

=========================
MemTotal: 62192 kB
MemFree: 2728 kB
SwapCached: 12172 kB
SwapTotal: 262136 kB
SwapFree: 127772 kB
=========================

I'll continue to update this blog post as the compilation proceeds, but it's already been going for 2 hours steadily...

Progress with Work

I've accomplished several things for work - that is several programs compiled for my new uclibc root for the arm boards / TS-7xxx series - including

• getting nfs-mounting to work
• building / testing OpenVPN
• building troll-ftpd (needs testing)

I've also started a document to release to the ts-7000 group on how to do this from scratch. The cool part is that all of the configuration data, patches, binpkgs, etc are simply stored in /tinygentoo/etc/portage /tinygentoo/etc/make.conf /usr/portage/local/overlay and /usr/portage/local/binpkgs.

Troll-ftpd was compiled all-right, but I can't sign in as root, even though I could with the TS distro. I think I might have to write another patch for ftpd.c so that it properly checks passwords without crypt().

I also re-built openssl, but it seems a little bit bloated with the gentoo installation, and with the new 'engines' structure. I'm hopefully going to get rid of all of that, because really, we don't need all of the ciphers and we definitely don't need any of the engines because we have absolutely no crypto-specific hardware. Have to figure out what's up with that, so i'm waiting to hear back on the openssl list.

Still need to create a custom ebuild for JamVM - or even get it to build for the arm under uClibc for that matter :p

Also need to create a custom ebuild for the GNU Classpath, considering 90% of it goes to waste on our boards (and it's just fat anyway).

Things on the To-Do List for work

I have to push a new filesystem and kernel, as well as installation instructions for work.

Currently, we're using the TS-7xxx boards, with a Cirrus EP93xx / ARM 920T processor, for our embedded devices. The trouble is, that the boards ship with a non-standard 2.4 kernel and the bloated glibc library. I've successfully compiled and installed the 2.6.20.6 linux kernel as well as a uClibc userland for the boards, but now I'm in the process of working out bugs and installing other software that is necessary for the basic functionality of our boards.

To-Do:

• Make nfs-mounting work on the boards (recompile uClibc with full RPC, and then busybox)
• Build OpenVPN
• Build an FTP service
• Build JamVM
• Copy over the GNU Classpath that we have
• Build Avahi
• (etc, etc)

You may have guessed that this isn't such a small job. Anyway, I'm going to try to do quite a bit of it today.

Ciao!