OCTOBER 23, 2013

Releasing BoutiqueHotel.me

Last week, me and Ted Valentin released our new website, BoutiqueHotel.me.

Boutique hotels are, for those who don't know, (usually quite small) hotels with thought-through concepts and unique attributes. You can for example check out boutique hotels in Stockholm or in Cape Town.

The site's tech stack is Python + Django & Celery, PostgreSQL with PostGIS as database, memcached for caching and Redis as Celery's backend/broker. All maps on the website are powered by wonderful Leaflet.

Hopefully, I will be be open sourcing some of the website's code through some JS/Python libs. For example our image viewer which calculates how to distribute the images to create a perfectly balanced photo album.

I've realised that there are a lot of really nice hotels, and my current favorite is probably Treehotel in Harads quite far up north in Sweden.

Our aim with this site is to have the largest collection of the world's boutique hotels, and to give a far better experience when searching for boutique hotels than any other site out there. For this reason, we've started with just releasing the site for Sweden, and Swedish boutique hotels, but in the upcoming weeks we will roll out more countries.


SEPTEMBER 17, 2013

My flask entry point file that uses docopt

TAGS: flask, python,

I've been using Docopt - a neat library that lets you define command line arguments in a good looking, human readable docstring - in the entry point file of some of my Flask projects lately, and I thought I should share it.

It's nothing super fancy, but useful in development as it allows me to optionally specify the interface and port of the server. Here's the entry point file I use for What is my ZIP?:

Whatismyzip.com Website

  web.py <interface:port>
  web.py -h | --help

  -h --help    Show this help

import re

from docopt import docopt
from raven.contrib.flask import Sentry

from whatzip import app

if __name__ == '__main__':
    # try to parse host and port, to listen to, from command line
    arguments = docopt(__doc__, help=True)
    listen_to = arguments.get("<interface:port>")
    app_kwargs = {}
    if listen_to:
        matches = re.match("([0-9A-z.-]+):(\d{1,5})", listen_to)
        if matches:
            app_kwargs["host"] = matches.group(1)
            app_kwargs["port"] = int(matches.group(2))

    # run app
    app.run(debug=True, **app_kwargs)
    sentry = Sentry(app)

Then I can start the development server using either:

python web.py

or, if I'm happy with the default interface and port, I can do:

python web.py

And in production I run it through gunicorn:

gunicorn -b$PORT -k gevent web:app

The final Sentry(app) row under the else: is there to set up Sentry logging for the app, and that whole else clause can be skipped if you don't want Sentry logging in production.

Hopefully this can be useful to someone else :).


DECEMBER 06, 2012

Heroku: Running multiple python processes in a single dyno using foreman

Recently I've made a few small python web projects that I've deployed on Heroku. For some of these projects I've needed to run asynchronous jobs through Celery.

Heroku allows you to run one free dyno (or actually they give you 720 free dyno hours per month, which corresponds to one dyno constantly running). This means that if you choose to run one web dyno and one worker dyno (celery in this case), you'll be charged for 720 dyno hours. However, if you have a very small project, or your're working on a project that hasn't been released yet, you can avoid this cost.

A heroku dyno is like a process, and in this process you can actually spawn new processes, as long as you stay within the limit of 512 mb ram (the process also only has one CPU core). Heroku suggests that you use foreman when you run your application on your local machine, but you can actually use foreman on heroku, in order to run multiple processes in a single dyno.


1. The first thing we'll do is to create a post_compile script in the bin directory of our project. In this script we'll add a few lines that will install foreman and make it available in our project slug. Create the file PROJECT_ROOT/bin/post_compile with the following content:

#!/usr/bin/env bash

# set terminal to UTF8 (otherwise `gem install foreman` fails)
export LANG=en_CA.UTF-8

# set up ruby env
mkdir .ruby-gems
export GEM_HOME=$PWD/.ruby-gems
export PATH=$PATH:$PWD/.ruby-gems/bin

# install foreman
gem install foreman

2. Next, we'll create a new procfile, in the project root directory, which we'll call ProcfileFree:

web: gunicorn -b$PORT -w 4 myapp.wsgi:application
celery: python manage.py celeryd -c 3 --beat

3. The third thing to do is to update the "real" Procfile that Heroku uses, so that it starts foreman with our new ProcfileFree. The content of my Procfile is:

web: env > .env; env GEM_HOME=$HOME/.ruby-gems env PATH=$PATH:$HOME/.ruby-gems/bin foreman start -f ProcfileFree

And that's all! After deploying the above, and scaling the web dyno to 1, Heroku will start foreman which will start the processes that you've defined in ProcfileFree (in my example this was gunicorn and celery through Django's manage.py).

Please note

This method can be very useful for small projects sometimes, but it will not be performant. If you get more traffic than it can handle, scaling dynos the normal way is definitely the way to go.

Also note that when you're only running a single dyno, Heroku will stop it if it hasn't received any web requests in a while. One possible way to get around this is to ping your website with http requests regularly.

Lastly, I just want to say that I think Heroku is a great service, and I don't mind paying for it at all, which I do for a few larger projects that I host there.


JULY 05, 2011

Extending templates from a specific Django app

TAGS: django, python,

A while ago (well, actually a long time ago since I had this blog post laying around as a draft, until I stumbled across a stack overflow question that this post would answer) I wanted to customize Django's admin app, by adding a link to a statistics page that I had built, on the index.html page of the admin app.

I could have copied the index.html from the templates directory in the admin app, to my project's templates directory and added my link, but this would have given me a maintenance task to keep Django's admin templates up to date in future versions. So what I really wanted to do was to define my own admin/index.html file in the project's templates directory, and then in my index template extend the index template from the admin app. However this isn't supported by Django be default.

When I searched the web I found all sort of solutions to the problem, some involving symlinking the admin's index.html to another path and then extending the symlink, but that wasn't an option for me since I develop on both Windows and Unix, and besides, it didn't feel like a very clean solution.

Finally I found very nice template loader on djangosnippets.org which solved my problem! It allows you to extend a template from a specific Django app by using the following syntax:

{% extends "admin:admin/index.html" %}

As you can see, I specify the app where the template resides before the colon. The admin/index.html, in my project's template directory, that I finally ended up with looks like this:

{% extends "admin:admin/index.html" %}

{% block sidebar %}
        < h1>Statistics< /h1>
        < a href="/admin/station/stats/">Published Stations< /a>
{% endblock %}

Now I won't have to change anything if the Django admin app's index.html is updated in a newer version of Django, unless the sidebar block is removed, or something similar.

Also, if you decide to use this trick, don't forget to add the template loader to the TEMPLATE_LOADERS list in your project's settings.py file.


APRIL 22, 2010

Python urllib2 timeout issue

TAGS: python,

I use urllib2 from Python's standard library, in quite a few projects. It's quite nice, but the documentation isn't very comprehensive and it always makes me feel like I'm programming Java once I want to do something more complicated than just open an URL and read the response (i.e. handling redirect responses, reading response headers, etc).

Anyway, the other day I found - if not a bug - then at least an undocumented issue. Since Python 2.6, urllib2 provides a way to set the timeout time, like in the following code where the timeout is set to 2.5 seconds:

import urllib2

    response = urllib2.urlopen("http://google.com", None, 2.5)
except URLError, e:
    print "Oops, timed out?"

If no timeout is specified, the global socket timeout value will be used, which by default is infinite.

The above code will catch almost every timeout, but the problem is that you might still get a timeout raised as a totally different exception:

File "/usr/lib/python2.4/socket.py", line 285, in read
  data = self._sock.recv(recv_size)
File "/usr/lib/python2.4/httplib.py", line 460, in read
  return self._read_chunked(amt)
File "/usr/lib/python2.4/httplib.py", line 495, in _read_chunked
  line = self.fp.readline()
File "/usr/lib/python2.4/socket.py", line 325, in readline
  data = recv(1)
socket.timeout: timed out

The solution is to catch this other exception, thrown by python's socket lib, as well:

import urllib2
import socket

    response = urllib2.urlopen("http://google.com", None, 2.5)
except URLError, e:
    print "Oops, timed out?"
except socket.timeout:
    print "Timed out!"

Hopefully this will save someone else some headache :).


MARCH 03, 2010

Favorite Django Tips

TAGS: django, python, web,

A few months ago I found a really useful Stack Overflow Question. Here are my favorites from the answers.

Use render_to decorator instead of render_to_response

This decorator is found in the app django annoying, and is a very nice shortcut for declaring what template a view should render.

Instead of returning the response of render_to_response, you just return a python dict which will be used as the template context for the template specified as argument to the @render_to decorator. If anything else than a dict is returned, normal view processing will occur, so this won't break redirects or any other cases where you might return a HttpResponse (for example normal render_to_response code).

Anyway, here is an example on how to use it:

def list_posts(request):
    posts = BlogPost.objects.all()
    return {"blog_posts": posts}

This equals to:

def list_posts(request):
    posts = BlogPost.objects.all()
    return render_to_response('list.html',
        {'blog_posts': posts},

Update (22/4): Marcin Nowak notified me that the render_to decorator breaks Django Reusable App converions, so I made a fork of django-annoying where I modified the render_to decorator to support template_name and extra_context keyword arguments.

Load custom template tags in all templates

Custom template tags that you use all over your templates can be auto loaded. Just add the following in a module that is loaded (i.e. your urlconf if you want the template tags to be loaded for the whole project)

from django import template

Use relative paths in settings.py

I hesitated about adding this tips, since I think it's quite obvious, but since so many people on Stack Overflow has voted it up, I guess there are people who use(d) absolute paths in their settings.py.

Don't use absolute paths in settings.py (i.e /home/jonatan/...), instead use relative paths so that the project will work wherever it resides.

import os.path
    os.path.join(os.path.dirname(__file__), "templates"),



Jonatan Heyman My name is Jonatan Heyman, and I'm a programmer. On this blog I write about programming, the web and technology. I also listen to a large number of indie pop tunes. You can read more about me on the about page.



Copyright (c) 2009-2010 Jonatan Heyman