Django on Leopard

Part of the reason I was waiting to switch from using Windows XP (gasp) as my Django development platform to OS X is that it is actually easier to install the platform on XP. Windows does not come with any of the tools, so it was a matter of installing the version that I need. For OS X, it had a older version of python, so I would have to have multiple versions of it running. But Leopard changes all that. So I switched! Running Django on os x becomes much easier with Leopard. Out of the box Leopard comes with almost everything I need :

Python is at version 2.5.1 • svn is at version 1.4.4 • SQLite is at version 3.4 with python library installed

For development, I use SQLite and the built in web server only. I run Postgres on the production boxes. Because I do not have to install Postgres, getting Django setup is just a matter of setting up the rest of the environment:

Install Django

You can just download the installation tarball and do a standard Python install. But to have a little more flexibility in switching Django versions, I download both the production and trunk versions of Django into my own directory. Then using a soft link I can switch between the two. Download Django into my own home directory, under extlib, where I put all the external libraries. Then link from the Python site-packages directory to my own Django directory. By switching where that link points, I can switch version.

# Make directory and download two versions of Django: cd $HOME mkdir extlib cd extlib svn co http://code.djangoproject.com/svn/django/tags/releases/0.96.1 django-0.96 svn co http://code.djangoproject.com/svn/django/trunk/ django_trunk

# Then symlink instead of installing the django code into the Python site-packages, which is at # at /Library/Python/2.5/site-packages on my machine.

sudo ln -s ~/extlib/django-0.96/django django

Note: You can find the location of the library by running a simple python statement:

python -c ' import sys; print sys.path'

and look for the site-packages directory.

Setup Shell

I need to setup my shell environment so that my (multiple) Django projects can find Django. If you need help in figuring out where to put things, bashrc vs bash_profile etc, this article explains how and where to put things very well. Most importantly, add the django bin directory to your path:

PATH=$PATH:/Library/Python2.5/site-packages/django/bin Note: I am adding the Django bin directory to the PATH via the python site-packages link. That way if I switch Django versions via the site-packages link, the PATH will switch automatically!

Python Image Library

I need to use this for image processing. I thought I could get away from installing the free Apple developer tools but I couldn't. I have to install the Apple XCodeTools set first, from the OS X installation CD. Then download the source files here .

Before running the install, you also need to get the libjpeg for JPEG support:

Dowload the source at http://www.ijg.org/files/jpegsrc.v6b.tar.gz untar it, cd into the jpeg-6b directory and then:

cp /usr/share/libtool/config.sub . cp /usr/share/libtool/config.guess . ./configure --enable-shared make sudo mkdir -p /usr/local/include sudo mkdir -p /usr/local/bin sudo mkdir -p /usr/local/lib sudo mkdir -p /usr/local/man/man1 sudo make install

The following the PIL build instructions:

edit the setup.py file, add the lines instead of setting them to None:

JPEG_ROOT = "/usr/local/include" ZLIB_ROOT = "/usr/local/include"

then run:

python setup.py build_ext -i python selftest.py sudo python setup.py install

Django 0.96 Internationalization

I am trying out Django's i18n support and found a few issues. This only applies if you are using newforms and Django 0.96. Order of Middleware

First, the order of the middleware is important. It is defined in the documentation, but I missed it the first time around. You have to put the localeMiddleware after SessionMiddleware:

MIDDLEWARE_CLASSES = ( 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', )

Newforms label

There is a known bug in the newforms' handling of label text in the 0.96 release. It loads the label text at load time. So if you switch language after a form is rendered, it will not re-evaluate the display text for the new language. There is a patch at ticket number 4904. Apply this patch to your newforms.util.py

Then make sure you define label text as gettext_lazy:

from django.utils.translation import gettext_lazy class MyForm(forms.Form): myfield = forms.CharField(label=gettext_lazy('translatable_text'))

Newforms Label Trailing Colon

Another little problem with newforms at 0.96. The newforms engine constructs default HTML display label from the form's field names and always append a colon to it. While in the development version, it first checks for a trailing punctuation mark first. If a supplied label already ends in a colon, it will not add another colon. Perhaps because of this, the current locale files have many Django labels defined, with and without the trailing colon. Kinda strange huh?

Using pytextile in Django – problem with unicode

I am coverting a J2EE application over to django. The existing app uses Textile as a simple markup for user text input. After installing pytextile (which is under ownership change at the moment), I found a problem using it: The "textile" filter works for strings and strings retrieved from my database, but when I try to use it on a piece of string that I received from a web form, it blew up with an encoding error. This is the issue:

• Django nicely uses unicode string internally. When Django receive a string from a web form, it converts it from the browser encoding (utf8) to unicode.
• pytextile expects the string to be in either the system default encoding, or when called in the filter, with the Django default charset (usually utf-8).
• In my case pytextile choked on its own glyphs replacement code when trying to do a replace of latex style quotes on the string.

The solution is to convert the unicode string from forms input back to utf-8 first before giving it to pytextile:

for_textile_str = my_form_input_str.encode('utf8')
print textile(for_textile_str) # will now work.

Ownership change:

According to this webpage, I quote:

A couple of weeks I announced that I was looking for a new maintainer for pytextile, since I'm way too involved with my research and pydap. Well, as of today Silas Sewell is the new maintainer of pytextile.

The Importance of the initial argument in Django newforms library

I like Django a lot. In fact I am in the middle of converting a large J2EE application to Django. Sometimes I run into a problem using it, and knowing Django, I know there is a good solution. To find the solution however, sometimes require very careful reading of the documentation. Case in point. The solution here is completely documented in the Django online documentation. But perhaps, at least to me, not so obvious.

The problem:

A typical usage: display a form to add some data to the application. I need to include a hidden field and populate it with a value (obviously, since it is a hidden field, the only value is going to come from the application).

If you simply put the value in the dictionary to the form's constructor in the "GET" code path of your view handler, you will likely trigger the form's validation logic, and complain if you have any other normal fields that are required field.

The Solution:

The solution is to use the "initial" argument to the form's constructor, not to use the bounded data dictionary. The documentation discuss the "initial" argument as part of the unbounded newforms support, and I (wrongly assumed) normally do not think of my forms are unbounded. But they are, in the GET part of the handler.

Code Sample:

Class MyForm(forms.Form):
    important_field = forms.CharField(max_length=32,required=True)
    my_secret_field = forms.IntegerField(widget=forms.HiddenInput(),required=True)

def my_view(request):
    if request.method == 'POST':
        # do post stuff...
    else:
        # GET, do this:
        form = MyForm(initial={'my_secret_field':42})
        # do NOT do this, because form will be displayed with
        # error: important_field is required on first display:
        # form = MyForm({'my_secret_field':42})

psycopg2 broken for 64 bit ubuntu

I want to test out amd64 bit ubuntu. Django keeps failing on syncdb. Turned out the python Postgresql library, psycopg2 is "severely broken" in the 64bit packaging. You will get a segment fault when trying to run the simplest thing like syncdb on an empty system if you are using postgresql. Oh well, for now, back to sqlite3.

Django dumpdata does not work with numeric fields

I try to dump my database in Postgres to json so that I can load it into my test system. Executing manage.py dumpdata gives me an error:

Unable to serialize database: Decimal("349.00") is not JSON serializable

Some googling finds ticket number 3324, a problem with the serializer. This is patched in the development tree already. Since I want to keep 0.96 in production, I just hand patched the json serializer source file from the diff, and it works now.

See the diff output here .

Django + Ajax != dojo

If you good for "django" and "ajax", you will quickly come across the "Django adopts Dojo as Ajax framework" article at Ajaxian. The important thing to note is that is it not (yet) true. Django 0.96, the current release, does not contain dojo. The dojo toolkit if definitely one of the better ones out there, but it is not part of the django distribution. Dojo itself is going thru a major change from 0.4x to 0.9. Perhaps it would make sense to wait until dojo is updated before making it the official ajax toolkit. Or, just let the developer pick her own.

Django Installation on Apache

I moved my Django environment from development to production here. One positive note for Django -- it is relatively easy to do the migration, and, I am running on different database and servers between the two environment. I am using the built-in development web server in development and sqlite as the database. In production I am using Apache and Postgres. I will write up how I migrate data between the two databases with relative ease in a separate article.

Of course, there are still some steps that are needed to setup the initial production environment. Here are some notes:

Postgres

I need to disable "ident" style authentication for the Postgres database connection between Apache's mod_python and Postgres. The normal/default postgres installation expects database connections made from normal linux users using OS authentication. Instead, from mod_python + Apache user, we need to enable plain user id + password authentication:

• edit the postgres hba authentication file. On my SUSE system it is at: /var/lib/pgsql/data/pg_hba.conf
• Add a line for my application database:
  local <my_database> all password
• Leaving the existing catch all for the other databases:
  local all all ident sameuser
• to reload this file, use pg_ctl reload (as postgres user)

Setup mod_python

Add the following block to your host file:

<IfModule mod_python.c>

<Location />
     SetHandler python-program
     PythonHandler      django.core.handlers.modpython
     SETEnv DJANGO_SETTINGS_MODULE desk.settings
     PythonDebug On
     PythonPath "['my_project_root'] + ['my_app_root'] + sys.path"
</Location>

<Location "/media">
     SetHandler None
</Location>

<LocationMatch "\.(jpg|gif|png)$">
      SetHandler None
</locationMatch>

</IfModule>

The first location block setups the python module to run Django. One thing of note is the setting of PythonPath. You should add BOTH your project root and your application root to the path. Otherwise you will have to reference the project name inside your applications, which is a no-no.

Linking Django admin files

Finally, don't forget to make a link to make the Django admin system works. You need to create a link in your apache document root to the installed Django admin file directory. e.g.:

cd my_doc_root
ln -s /usr/lib/python/site-packages/django/contrib/admin/media media

Note that Django uses the name "media" to reference those files. Since most likely you want to put your static files in the same directory structure, you have to name your static file directory something other than Django.

Custom Validation in Django newforms library

This is how you add your own additional validation logic to a form in the newforms library:

Create a clean_XXXX method in your forms class. This will be called by the forms validation logic. Subsitute XXXX with the field name. Inside this method you will have access to all forms data clean’ed so far, including the field in question. After performing any custom validation, either returned the cleaned and validated data, or raise ValidationError.

Note: The ValidationError type is now from django.newforms.util, not from django.core.validators. Be careful with that.

This is the typical user registration type form as an example:

from django import newforms as forms
from django.newforms.util import ValidationError

class RegForm(forms.Form):
     username = forms.CharField( max_length=30),
     email = forms.EmailField(max_length=256),
     password1 = forms.CharField(max_length=16,widget=forms.PasswordInput)
     password2 = forms.CharField(max_length=16,widget=forms.PasswordInput)

     def clean_username(self):
          n = self.clean_data[’username’]
          try:
                  User.objects.get(username=n)
          except User.DoesNotExist:
                  return

          raise ValidationError(’The username “%s” is already taken.’ % n)

     def clean_password2(self):
          if self.clean_data[’password1?] != self.clean_data[’password2?]:
               raise ValidationError(’Passwords must match.’)
          else:
               return self.clean_data['password2']

There are more examples in the test script as well.