Linux Fu: Fusing Hackaday [Hackaday]

View Article on Hackaday

Unix and, by extension, Linux, has a mantra to make everything possible look like a file. Files, of course, look like files. But also devices, network sockets, and even system information show up as things that appear to be files. There are plenty of advantages to doing that since you can use all the nice tools like grep and find to work with files. However, making your own programs expose a filesystem can be hard. Filesystem code traditionally works at the kernel module level, where mistakes can wipe out lots of things and debugging is difficult. However, there is FUSE — the file system in user space library — that allows you to write more or less ordinary code and expose anything you want as a file system. You’ve probably seen FUSE used to mount, say, remote drives via ssh or Dropbox. We’ve even looked at FUSE before, even for Windows.

What’s missing, naturally, is the Hackaday RSS feed, mountable as a normal file. And that’s what we’re building today.

Writing a FUSE filesystem isn’t that hard, but there are a lot of tedious jobs. You essentially have to provide callbacks that FUSE uses to do things when the operating system asks for them. Open a file, read a file, list a directory, etc. The problem is that for some simple projects, you don’t care about half of these things, but you still have to provide them.

Luckily, there are libraries that can make it a lot easier. I’m going to show you a simple C++ program that can mount your favorite RSS feed (assuming your favorite one is Hackaday, of course) as a file system. Granted, that’s not amazing, but it is kind of neat to be able to grep through the front page stories from the command line or view the last few articles using Dolphin.

Pick a Library

There are plenty of libraries and wrappers around FUSE. I picked one by [jachappell] over on GitHub. It was pretty simple and hides just enough of FUSE to be handy, but not so much as to be a problem. All the code is hidden around in Fuse.h.

One thing to note is that the library assumes you are using libfuse 3.0. If you don’t already have it, you’ll have to install the libfuse 3.0 development package from your package manager. There are other choices of libraries, of course, and you could just write to the underlying libfuse implementation, but a good library can make it much simpler to get started.

Just to keep things simple, I forked the original project on GitHub and added a fusehad directory.

Constraints

To keep things simple, I decided not to worry about performance too much. Since the data is traveling over the network, I do attempt to cache it, and I don’t refresh data later. Of course, you can’t write to the filesystem at all. This is purely for reading Hackaday.

These constraints make things easier. Of course, if you were writing your own filesystem, you might relax some of these, but it still helps to get something as simple as possible working first.

Making it Work First

Speaking of which, the first order of business is to be able to read the Hackaday RSS feed and pull out the parts we need. Again, not worrying about performance, I decided to do that with a pipe and calling out to curl. Yes, that’s cheating, but it works just fine, and that’s why we have tools in the first place.

The HaDFS.cpp file has a few functions related to FUSE and some helper functions, too. However, I wanted to focus on getting the RSS feed working so I put the related code into a function I made up called userinit. I found out the hard way that naming it init would conflict with the library.

The normal FUSE system processes your command line arguments — a good thing, as you’ll see soon. So the main in HaD.cpp is really simple:

#include 
#include "HaDFS.h"
int main(int argc, char *argv[])
{
  HaDFS fs;
  if (fs.userinit())
  {
    fprintf(stderr,"Can't fetch feedn");
    return 99;
  };
  int status;
  status= fs.run(argc, argv);
  return status;
}

However, for now, I simply commented out the line that calls fs.run. That left me with a simple program that just calls userinit.

Reading the feed isn’t that hard since I’m conscripting curl. Each topic is in a structure and there is an array of these structures. If you try to load too many stories, the code just quietly discards the excess (see MAXTOPIC). The topics global variable tells how many stories we’ve actually loaded.

// The curl line to read our feed
static char cmd[]="curl https://hackaday.com/feed/ 2>/dev/null | egrep '(