Angled and More

Well, looks like it’s time for the annual “great summer” report!

I managed to finish my GSoC project for Plan 9 from Bell Labs this year well in time, and I’m quite pleased with myself. I learned so much, and got to spend time with some of the most brilliant people in the field of computer science… it’s been so amazing that I’m at a loss for words.

I had so much fun implementing the 9P protocol in JavaScript that I almost immeditey began a native PHP implementation; right now I’m polishing it for inclusion in PEAR. And hopefully, someday, my bindings over libixp might get included as a bundled extension in the PHP distribution.

Another cool thing I managed to do this summer was build a firefox extension. Nothing too fancy, but I’m still happy with what I managed to do. Times like this, I can’t help but bow to the IRC and Usenet overlords – couldn’t have done it without the help of those dozens of people!

After working with Plan 9 for a while, I got this crazy idea of trying to build a toy OS of my own, just for kicks. Really, looking at the source code for Plan 9 makes you believe that you don’t have to be a rockstar to build an operating system. Sure enough, after some help from the folks at osdev, I managed to get “Hello World!” on the Parallels virtual machine screen. I feel good.

Freed.IN is coming up (formerly known as Freedel), and I sent in a proposal on creating synthetic file-systems with (guess what) 9P. Irrespective of whether my talk is chosen or not, I’m going to Delhi; there’s too much fun involved to even think about missing :)

Oh, and, I got myself a job (tee-hee) at the Computer Sciences Corporation. But, well, Yahoo! is visiting our campus on the 4th… let’s see what the future holds for me.

Also, if you’re wondering why I haven’t posted in a while, you’re reading the wrong feed. Allow me to point you to my aggregated feed – you’ll see that I have, infact, been posting stuff whole summer :)

Say Hello to Angled 0.1!

I feel like I’m in the seventh heaven. After a few sleepless nights struggling with Mozilla’s XPCOM, I finally got the 9P Firefox plugin to work.

The plugin is called Angled (an anagram of Glenda, the Plan 9 bunny) and is in a pretty simplistic state right now: you can read any files served by 9P right in your browser window. Let’s take a step by step look.

First I startup Inferno to start a 9P server. ${INFERNO}/usr/anant/home is symlinked to my actual home directory, /Users/anant:

$ emu
; runas nobody {listen -A tcp!localhost!1564 {export /usr/anant/home &}}

Let’s see what files are actually there:

[theghost web9]$ pwd
/Users/anant/Plan9/web9
[theghost web9]$ ls
README TODO   js9p   php9p

Alright, I open my browser window and type ‘ninep://localhost!1564/Plan9/web9/README’ into the address bar. I could also say ‘tcp!localhost!1564′, but TCP is the only protocol available for Angled, so it would be redundant. Now, for the goodies: Screenshots!

Read Text files over 9P

Cool! But wait, Angled also displays binary files right in the browser. There’s a catch though, it will only work for binary files that can be viewed directly in the browser window. Certain types of files (.doc for example) do trigger a download request, but then become corrupted for some reason.

[theghost content]$ pwd
/Users/anant/Plan9/web9/js9p/angled/content
[theghost content]$ ls
angled.png         firefoxOverlay.xul glenda-error.png   overlay.js

Let’s say I want to view angled.png. Here’s what I get:

Angled shows Images too

Okay, but what if you type in a URL that points to an invalid file? Check this out:

Errors in Angled

Sweet! I’m yet to figure out how to transmit the exact error message to that page, so you’ll have to make do with that generic image for now.

Okay, now onto the bad parts. Angled doesn’t support authentication yet (although the base JS implementation is capable of generating and parsing auth messages). Next, you won’t get directory listings (you’ll get a bunch of binary gibberish which is actually Rstat messages for the directory’s contents). Also, I’m doing the 9P connection and transactions in a blocking thread, so the UI freezes while all that is done. I couldn’t feel the difference since I was testing on my local 9P server, but connecting to remote 9P servers won’t be a pleasant experience. The solution to this is to create a custom nsIChannel implementation, which is a lot of work… I’ll do it when I get to it ;)

Enjoy!

Implementing a Protocol in Mozilla

Creating a Firefox extension is nothing short of an adventure. I was able to get started pretty quickly, thanks to this web-based quick-start wizard, all the boilerplate code was generated in literally no time.

Now, onto the actual functionality of the extension. I have to implement a protocol handler for the 9P protocol, which essentially means you type in “ninep://sources.cs.bell-labs.com/” and start reading files right off the browser window. (ninep:// because a URL can’t start with a number)

This page provides some useful insights and code snippets on the subject of adding a new protocol handler. I was able to get as far as displaying a Glenda image whenever you type in a URL beginning with ‘ninep’.

The way this works is you create an XPCOM component that implements a standard interface. Specifically, the newChannel() method is where all the action is. It receives a URL and you do something and return an nsIChannel. Mozilla provides standard nsIChannel implementations for popular protocols such as http, ftp and even the ubiquitous file://.

The intuitive thing to do here would be to do all my 9P processing in the newChannel() implementation and return a stream in a standard channel. However, that’s not going to work, since newChannel() would then block and the UI would actually freeze until the 9P transaction completes. Sub-optimal.

The “proper” way to do this would be to create my own implementation of nsIChannel. That way I just create a new nsIChannel in newChannel() and be on my way. nsIChannel would then take care of firing callbacks as and when data arrives. There’s somewhere I can start with, and that’s the Mozilla implementation of the finger protocol. It’s written in C++, however, and I need to figure out how I can map the same to JavaScript (via XPConnect).

Fiddling with Binary in JavaScript

Since ordinary JavaScript cannot directly communicate with a 9P server (over TCP), we decided to go in for a 2-tier approach: A script on the client generates a 9P message which is sent to a server via an XMLHttpRequest. The server then forwards the message to the actual 9P server. Messages from the 9P server to the client are sent in a similar fashion.

All 9P messages are just binary sequences, which means I need some way of representing a 9P message in JavaScript. A character is always 1 byte, so representing characters is not a problem. For representing integers in binary I use the following snippet of code:

while(num) {
  str[str.length] = String.fromCharCode(num % 256);
  num = Math.floor(num / 256);
}

where ‘num’ is the number to be encoded, and str is returned as: str + join(””).

This means I can now encode any 9P message as a simple sequence of JavaScript strings. The final string can then be sent along with the payload of an XMLHttpRequest. I’m wondering whether it will be a good idea to encode the string in Base64 first, although an XMLHttpRequest should have no trouble with the string representation either.

Mapping 9P to REST

Now that the PHP bindings to libixp are somewhat usable, I’ve moved on to the JavaScript portion of my project. The first (and easier!) part of it is to map 9P to a RESTful scheme, so traditional web developers can use 9P without having to learn anything new.

The PHP bindings to libixp was an important pre-requisite to achieve this: presenting a REST interface to an existing 9P serve would require a “bridge” at the middle, to convert REST requests to 9P requests and vice-versa with responses. This “bridge” may be present at any location that is mutually accessible by the client wanting RESTful access and the server providing the 9P service. This bridge can be easily coded in PHP using the new libixp bindings.

To those not very familiar with REST, it is simply a way of accessing resources using plain-old HTTP. It’s become quite popular with web developers these days, as a much simpler alternative to SOAP. A lot of web services these days are RESTful, including those offered by Amazon and Yahoo.

From the client’s perspective, accessing a 9P resource boils down to 4 things: reading, creating, modifying and deleting. These operations map neatly onto the GET, PUT, POST and DELETE HTTP requests, respectively. And thus, we have our REST URI scheme. This scheme would be enough if my bridge exposes only a single 9P serve as a REST service. As an example, suppose I start a bridge at http://plan9.kix.in/rest/ that exposes only the tcp!sources.cs.bell-labs.com!564 9P serve, to read the file named ‘lsr’ I would perform a GET request at http://plan9.kix.in/rest/lsr.

However, the plot thickens when I want to design a bridge that allows access to any 9P service (which is definitely better). Now we need to encode the information represented in Plan 9 as: tcp!sources.cs.bell-labs.com!564 into a HTTP URI. The intuitive thing to do would be something like: [ROOT]/[PROTOCOL]/[9P-URI]/[PORT]/[FILE-PATH].

Hmm. That leads to really long URI’s like http://plan9.kix.in/rest/tcp/sources.cs.bell-labs.com/564/lsr. Besides that, there are several things that need to be worked out. What happens when you do a GET on a file that is actually a directory? What about parameters to GET that you usually pass to a read() function: Suppose you want to read the first 1024 bytes of a file only?

Comments and Suggestions welcome :)

P.S.: Thanks to some pointers by Kris, the PHP9P client example shown in the previous post now handles binary files correctly.

Herald the PHP 9P Client

I’ve given a few final touches to the 9P client for PHP. All the basic stuff should work (if they don’t please let me know!), and I’ve written a script that shows some of the basic functionality that the client offers. Two actually, this one for the CLI SAPI and this one for the Apache2 SAPI. All they do is read files and show directory listings, but the scripts are a good place to see what the API is like, since there’s no official documentation yet. I couldn’t locate a proper PHP 5 server to actually host the second example, which would have been cool… but now we’ll just have to make do with these screenshots I took off my browser (with the script running on my local Apache):



Startup Screen


Once you’ve put in the address, you’ll get a directory listing of the root:



Viewer


While the CLI SAPI demo script also handles binary files quite well, this one will just send gibberish to your screen if you click on a binary file (text files work fine though). I’m still trying to figure out what’s the best way of determining the MIME type of a file so I can send the appropriate HTTP header before transmitting the data itself,  so your browser would know how to interpret it. As for the server-side of things, the code is still in a state of flux and I’m not decided on what kind of API to offer. I’ll probably take a break from this part of the project and move on the JS bindings, and then come back to tie this up in the end. That’s not to say that the server-side code is not usable, just that it’s not very developer-friendly; feel free to play around with it if you’re feeling adventurous :)

Follow

Get every new post delivered to your Inbox.