student post feed the hard way

I decided for the Spring semester to write my course site using Flask, a python micro web framework. By saying it is a micro framework, I’m saying it doesn’t do near as much for you as Ruby on Rails and the like. I haven’t written a web app from scratch before, so I figured it would be a good exercise to do so for a course site, especially since I want to do something much more complex for next Fall’s Seminar on Digital History in Theory and Practice.

Flask is great, from my perspective, for a couple of reasons. 1. It’s in python. (I’m comfortable with python, so I don’t have to learn a bunch of new language conventions while also learning to build the web app.) 2. Even though as a microframework, Flask doesn’t handle a ton for you, there are already a bunch of extensions out there that help quite a bit.

The past two years or so, I’ve been building my course sites with hyde, a python static site generator much like ruby’s jekyll. I think that most of what we do with course sites can be done statically, with just a little client-side javascript. And, in fact, there is nothing on this new site that I couldn’t do that way.

So, why bother with Flask then? I don’t know. Shits and giggles? The chance to learn how to deploy a wsgi app on Dreamhost? Learning at a very basic level about routing views, app configs, etc. I still didn’t want to mess with a SQL database for the site, because other than tracking student posts there’s no point in it. Everything else my course sites need, at least for now, are better served as, essentially, static files.

So, individual pages for the site are all written in Markdown, and served using a flask extension named flat-pages. It’s a nifty plugin. When the app receives a url request, the path is checked against a folder named pages. So, for example, checks for a file named syllabus. Since it’s there, it takes the text of that file, which is still in Markdown, converts it to an html snippet and serves it through a page template. This is, in many ways, what a static site generator does, and what Frozen Flask does to produce static sites from a Flask app. (I’m going to do that when this course is over. There’s no need for an in-active site to be anything but a static site.)

This brings us back to a page of student post feeds. I require my students to each start a blog for my course. They write weekly, and I always have a place on the site that aggregates their feeds. There are many easy ways to do this. For my previous static sites, I’ve used a number of javascript options. For my personal site, I use feedjs for an up-to-date feed of this site. For course sites I’ve been using Google’s Feed API, which lets me embed a whole bunch of feeds into a singe frame. It’s also nice because Google caches feeds and lets you control a number of features. Sharon Leon is using a Google Reader bundle for The Digital Past, which is very nice given the ease of putting together and sharing bundles in Reader. I always actually track student posts for grading purposes that way.

But all of that is too easy. For my Flask site, I first put the feed retrieval directly in site logic with a function that checked feeds each time someone visited the student work page. By the way, I’m using a python package named feedparser for getting the feeds. This works. But, it’s also very slow and requires a whole bunch of resources to do retrieve things that are rarely updated. A better solution would be to store student feeds in an SQL, or even NoSQL database. But, I only want to show the most recent post on the site, so all the caching and checking, and then the overhead of adding an ORM to the app seemed like more trouble that it’s worth.

Solution? Why not write a script that gets the most recent post of a student site and have it directly write Markdown to a file in the pages directory? So, that’s what I did. I run the script with a cron job. Speaking of which, that’s another thing I learned doing this– how to write a crontab entry and deal with script path problems and the like.

feedparser returns a dictionary-like object that is fairly complex, so it took a bit of time figuring out where the elements of the feed were that I wanted to display. Then, I had encoding problems. Because Python 2.x always ends up in an encoding problem. feedparser returns unicode objects, and Python’s normal file writer tries to encode everything as ASCII.

The result is this page, which is essentially programmatically-written Markdown. All the code for the site is on github.

Next semester I’m teaching a grad/undergrad seminar on Digital History. That site will also be a Flask app, but this time I’m going to have students write posts directly on the site through an admin interface. Yes, I’m going to reinvent the blog, because doing it the hard way is better for me.


Associate Professor of Early Latin America Department of History University of Tennessee-Knoxville

Tagged with: , , ,
Posted in Uncategorized
2 comments on “student post feed the hard way
  1. Great, now you’ve identified another rabbit hole for me to go down …

    By the way, the link at the bottom of the post for “this page” is broken. Missing the course number, looks like.

  2. ctb says:

    Oops. Fixed the link. Thanks, Caleb.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Hacer juicio ú dictamen acerca de alguna cosa... significando que el objeto excita el juicio ú dictamen en la persona que le hace.

Deducir ante el Juez la accion ú derecho que se tiene, ó las excepciones que excluyen la accion contrária.

RAE 1737 Academia autoridades
Buy my book!

Chad Black

I, your humble contributor, am Chad Black. You can also find me on the web here.
%d bloggers like this: