Tag: django

When using Sphinx with Postgresql you have to be a little careful. Other than the search binary requiring the MySQL libraries even when you are not using MySQL, I just encountered another issue with Postgresql where it doesn’t look at postgresql boolean attributes correctly; in this case sphinx thinks of all values as true. To get around this, until the fix is released, you can CAST the boolean field as integer. So, in your sql_query you would do something like the following.

bc[sql].
sql_query = SELECT id, title, DATE_PART(‘epoch’, created_on) AS created_on, CAST FROM mytable

The above SQL also demonstrates the DATE_PART function. The sphinx documentation shows UNIX_TIMESTAMP, which is a MySQL only function, as the function to use when indexing sql_attr_timestamp.

Python has a nice module called doctest that lets you embed test cases with the function documentation. This is a really nice time saver because these test cases also serve as examples, which I would have to write anyways. I was working on a function where I decided to write a few doctests, but I did not like waiting for django’s test runner to create the database and then run the tests, which wasn’t necessary for these specific tests. So, I thought why not manually run the doctests in a django shell?

ipython has support for outputting in doctest mode (%doctest_mode magic function), but it doesn’t come with any magic functions that will let you quickly run doctests. So, I ended up writing the following magic function that will let you accomplish this.

To use this, here is what you do.

1. Edit your ~/.ipython/ipy_user_conf.py file…
   1. Copy and paste the dodoctest function at the bottom (before the main() call).
   2. Copy and paste the ip.expose_magic call all the way at the end of the file (i.e. after the main() call).
2. Run django shell (or ipython directly): django-admin.py shell
3. Import your models: import myproject.myapp.models
4. Call the %dodoctest function with your models as a parameter:

   %dodoctest myproject.myapp.models

The magic function.

1
2
3
4
5
6
7
8
9

def dodoctest(self, arg):
    """Initializes the test runner and runs the doc tests for the specified object."""
    ip = self.api
    ip.ex("""import doctest, unittest;
suite = unittest.TestSuite()
suite.addTest(doctest.DocTestSuite(%s))
runner = unittest.TextTestRunner()
runner.run(suite)
""" % arg)

The call to register the above magic function.

1

ip.expose_magic('dodoctest', dodoctest)

As I was working on my website in Django I had a need for ordering a model based queryset on a custom SQL. Searching around the net I found this entry in the Django user group, but according that entry I have to create a suclass of the queryset and then a manager, in addition to other things. I didn’t like that solution so I continued looking around and finally came up with one that I think is cool and accomplishes the goal. Basically, we have to use the extra() method to create a new column based on custom SQL and then use that in the order by clause.

1
2

comments = Comment.objects.filter(node__id=node.id).extra(
  select={'thread_order': "SUBSTR(thread, 1, LENGTH(thread)-1)"}).order_by('thread_order')

In this case I needed to exclude the last character of the ‘thread’ column, and that’s what the SUBSTR does. So, there you have it, a quick way to order model querysets by custom SQL.

Bottom line: If you keep getting “No JSON object could be decoded” when loading JSON objects in django that look perfectly valid then make sure that you are using ASCII encoding.

I have been experimenting with django’s test framework and I recently hit a wall while creating a JSON fixture. Django wouldn’t like my JSON file and keep spitting out…

Problem installing fixture 'thebitguru/nodes/fixtures/nofmt_nodes.json': No JSON object could be decoded

The fixture was very simple and for the longest time I couldn’t figure out what was wrong. I had used the code>dumpdata command to dump the JSON in the first place, but after Magus- on #django (IRC) asked me if that’s what I had done, I decided to do it again. This time somehow magically it worked! I used TortoiseMerge to diff the two files and it claimed that they were exactly the same. Obviously, they weren’t!

I, being I who I am, wasn’t satisfied without finding out really why my initial export was not working. So, off I went to do some investigation. After about fifteen minutes and several different hex dumps and google searches I was looking at this wikipedia entry. Yes, 0xFEFF! That was the key to this puzzle. So, somehow my initial export had ended up in UTF-16 Big Endian encoding. A quick encoding change in VIM, set fileencoding=ascii, and everything was back to normal.

I took a trip down the memory lane and realized that I was using PowerShell at the time, and considering that it is fairly new and fancy shell I guessed that it must probably is using Unicode as the default output. If you look through the PowerShell User Guide you will see this specifically mentioned…

By default, the Out-File cmdlet creates a Unicode file. This is the best default in the long run, but it means that tools that expect ASCII files will not work correctly with the default output format. You can change the default output format to ASCII by using the Encoding parameter:

PS> Get-Process | Out-File -FilePath C:\temp\processlist.txt -Encoding ASCII

Another quick test verified that this was in fact the cause. Whew! Finally, another mystery solved!

I have been learning about django over the past two weeks or so. I am finally up to the point where I have started learning about its suggested test frameworks. The creators suggest using unittest, which is the standard python unit testing framework. I have liked this aspect of django: even though the core framework was built from ground up, many of its pieces still rely on standard python libraries.

Read full post… “Using custom settings in django tests”

Continuing with my theme of tips, about what might seem like pretty much random stuff, today I am going to show how to setup your PowerShell profile to allow you to run python and django-admin in the active shell. Without this if you run django-admin using something like python django-admin.py then it will pop up a new command prompt window. Basically, you need to do two steps.

1. Run PowerShell (as administrator in Windows Vista) and allow execution of local scripts without signing it: set-executionpolicy remotesigned
2. Create UserProfile\Documents\WindowsPowershell\profile.ps1 (on Vista or UserProfile\My Documents\ in other versions) and put the following text in there, changing the paths as appropriate.

$env:PYTHONPATH=“R:\django\packages\django-trunk”
$env:PATH=$env:PATH + “;D:\Programs\Python25\;R:\django\packages\django-trunk\django\bin”


	
function django-admin {python (gcm django-admin.py | resolve-path) $args}

If you are curious about the execution policy then checkout the very useful Getting Started document that came with PowerShell (look in your Start Menu).

2008-03-31, Update: I had an issue with the way I had posted this originally. I needed to add ‘$args’ so that the arguments would be passed to the script.