ld does not honor /etc/ld.so.conf?

[vda]

While slowly rebuilding Slackware linux to my taste I decided to
keep all apps separate under [/usr]/app/bar-1.2.3 style dirs,
including their /lib subdirs if any.

So far it works just great, but I have a problem with ld:
although I added /usr/app/qt/lib to /etc/ld.so.conf and reran
ldconfig, linking against qt libs fails.

man ld says that /etc/ld.so.conf directories will be searched for libs 
specified in -l<libname> options, it is apparently not true on my system.

Running shared binaries OTOH succeds. ldd <appname>
shows that it will successfully use a lib from /usr/app/qt/lib
if it needs qt.

(qt is just an example, I have same behavior with other libs too)

Is it a ld bug, manpage bug? Or am I doing something silly?

Please CC me
--
Denis Vlasenko