A few months ago I tried to setup django with Postgresql in Windows through cygwin. Part of this setup included installing the pyscopg2 module for python. Interestingly everything would compile and install OK, but when it came time to import the module, python would complain that it couldn’t find a file, specifically _psycopg.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/site-packages/psycopg2/__init__.py", line 60, in <module>
from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: No such file or directory
At first I decided to let it slide and do away with cygwin and go the Windows python route for the normal development, and using my “500MHz overclocked to 555MHz linux machine” to do advanced stuff. Lately this solution has been painful because of the background jobs has a significant difference in processing time; by significant I mean 22 minutes vs 5 minutes! So, over the past few days I decided that I was going to find out the solution to this. I started my investigation the normal way, doing some research on my own, checking in the IRC channels for python, asking on the psycopg mailing list, but no luck!
If you are only interested in the solution then please scroll down to the “Summary” section below. If you are interested in how I actually figured this out then continue reading…
Finally after ending at dead ends from all these sources I decided that I would really dive into this and figure it out on my own. Luckily after three hours of hard investigation, research and debugging I found out the cause. As a result of this adventure now I feel much better prepare for future issues related to cygwin. So, you are probably wondering about the solution… and here it is.
Even though python says the there is “No such file or directory,” I knew that the DLL existed in that exact location. This was made even more apparent by the fact that ‘python -v’ showed the dlopen() call to that exact path. The problem with cygwin (or possibly Windows; I can’t tell) is that it quietly errors out if a DLL fails to loads, in this case because of another missing DLLs. The way you find the missing DLLs is by using the handy ‘cygcheck.exe’ program. Let’s see what it says… Note that you have to pass the complete path, that’s what the `pwd` is doing
TheBitGuru@mystic /usr/lib/python2.5/site-packages/psycopg2$ cygcheck.exe `pwd`/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin/lib\python2.5/site-packages/psycopg2/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygwin1.dll
C:\Windows\system32\ADVAPI32.DLL
C:\Windows\system32\ntdll.dll
C:\Windows\system32\KERNEL32.dll
C:\Windows\system32\RPCRT4.dll
Error: could not find cygpq.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\libpython2.5.dll
So, that’s the problem! I am missing cygpq.dll. I knew that ‘pq’ is a Postgresql library so I fired up the setup cygwin.exe and looked for libpq. Interestingly, in my case libpq5 was already installed, but knowing that cygwin wasn’t finding cygpq.dll I went ahead and reinstalled it. After the process finished, I did another cygcheck…
TheBitGuru@mystic /usr/lib/python2.5/site-packages/psycopg2$ cygcheck.exe `pwd`/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin/lib\python2.5/site-packages/psycopg2/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygwin1.dll
C:\Windows\system32\ADVAPI32.DLL
C:\Windows\system32\ntdll.dll
C:\Windows\system32\KERNEL32.dll
C:\Windows\system32\RPCRT4.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygpq.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygcrypt-0.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygcrypto-0.9.8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygintl-8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygiconv-2.dll
Error: could not find cygldap-2-3-0.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygssl-0.9.8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\libpython2.5.dll
Hmm… this time I am missing cygldap, I have no idea where that is 🙁 So, I went an asked in #cygwin on IRC, and was pointed to the Setup Package Search link on cygwin homepage by “somian” (thanks!). A quick search for cygldap revealed that this was part of libopenldap. I installed that and tried again.
TheBitGuru@mystic /usr/lib/python2.5/site-packages/psycopg2$ cygcheck.exe `pwd`/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin/lib\python2.5/site-packages/psycopg2/_psycopg.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygwin1.dll
C:\Windows\system32\ADVAPI32.DLL
C:\Windows\system32\ntdll.dll
C:\Windows\system32\KERNEL32.dll
C:\Windows\system32\RPCRT4.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygpq.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygcrypt-0.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygcrypto-0.9.8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygintl-8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygiconv-2.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygldap-2-3-0.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cyglber-2-3-0.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygminires.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygssl-0.9.8.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\cygsasl2-2.dll
C:\Users\TheBitGuru\Programs\Cygwin\bin\libpython2.5.dll
Look at that! Excitedly, I started python to see the results…
TheBitGuru@mystic /usr/bin$ python Python 2.5.1 (r251:54863, May 18 2007, 16:56:43) [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import psycopg2 >>> psycopg2 <module 'psycopg2' from '/usr/lib/python2.5/site-packages/psycopg2/__init__.pyc'> >>>
Excellent, we are good to go!
Summary
To get psycopg2 to work you need to install libpq5 and libopenldap from the cygwin repository (using the normal cygwin setup). If you already have these two packages then try reinstalling these before going any further (once again using the normal cygwin setup). If the module still doesn’t work then I suggest you read this complete post and try to follow similar debugging steps to see if you can find the source of your issue. Good luck!