JNDI Lookup code quirks
I've seen multiple issues with not finding JNDI resources. On Websphere (I know, you don't use it, but it's good to know...) you'll have problems with
Context envCtx = (Context) initCtx.lookup("java:comp/env");
Session session = (Session)envCtx.lookup("mail/Session");
What works (on Websphere) is
Session session = (Session) initCtx.lookup("java:comp/env/mail/Session");
From what you wrote in another answer I understand that this is not your problem - I'm leaving it here for people coming across the same problem under different circumstances later.
Lookup of JNDI Resources from self-spawned threads
Also, access to JNDI resources might depend upon the thread looking up the resources. As far as I remember, threading is not well defined in the servlet api (or JEE or related areas. It might even be voluntarily and explicitly be non-defined.) Therefor it's not mandatory for the server to provide JNDI resources in threads that you've spawned yourself (again Websphere did bite me with this, I haven't tested Tomcat6 in this regard, earlier versions used to provide JNDI resources to all threads)
You've written that you are looking up JNDI resources from a singleton. If you, at the point of looking up resources, examine the stacktrace (either in your IDE, by throwing an exception or messing with Thread.currentThread().getStacktrace()): is there any Tomcat connector in the stacktrace or is one of your own Thread's run() method at the root of the stacktrace ? This would answer the question behind Jacks question if you are making the lookup from an Action class. (see Jack's answer)
Threads and call-environment part two
Create a new Struts Action and call your JNDI lookup code from there, see if this works if placed as close to struts and within the processing of a http request. If it works here, continue with the steps above.
validity of web.xml
Also, you might want to have a look at the schema definition for web.xml to make sure your resource-ref is positioned correctly. The servlet spec 2.4 is available at jcp.org and should be sufficient to check, even if tomcat implements 2.5.
After all, I believe that tomcat6 validates web.xml, so that you probably already have it at the correct position. (Can't remember, as my IDEs editor complains when I get it wrong, should I need to write a web.xml)
Tomcat debug options
Many context.xml entries honour the attribute 'debug'. Though I believe that low one-digit values are sufficient, I've adopted the habit to add 'debug="99"' to elements like 'Context' or others. You might want to see if this results in some helpful log entries.
Make sure it's not the classpath
As you seem to have fundamentally changed the environment, make sure that you have all required libraries on board - the mail api consists of several jars. Download a new copy and unpack all libraries to $CATALINA_HOME/lib. You might take unused away once it worked with all libraries in there.
Regarding your classpath question:
The reason for finding the class when put into $CATALINA_HOME/lib is, that the connection is done by the server (remember - you've defined it in context.xml which is read by the server in order to start the application), thus the jar must be on the servers classpath - not just on the applications (see the tomcat class loader HOWTO for more information)
EDIT:
Regarding your partial solution:
$CATALINA_HOME/conf/context.xml contains the global "default" context elements. This is not where you want your application specific configuration to be.
The standard position for tomcat is either in the webapps META-INF/context.xml or in an xml file (named as you like, ending .xml) in $CATALINA_HOME/conf/Catalina/localhost/. The later solution is actually prefered with regards to META-INF/context.xml because this way the configuration is independent of the application and doesn't get overwritten when a new application is deployed.
This context usually contains additional attributes like docBase and path:
<Context docBase="/location/where/you/deployed/your/application" path="/app">
...
</Context>
This way your application is available at http://servername:8080/app . If you deploy to the $CATALINA_HOME/webapps directory, the docBase value can be relative to webapp. Be careful with regards to race conditions though: Tomcat will autodeploy applications in $CATALINA_HOME/webapps and might create the context file. Also, deleting the webapp in order to deploy a new one might cause tomcat to delete the xml config file.
Therefor - whatever your problem might be: Try if your context definition / application work when placed in $CATALINA_HOME/conf/Catalina/localhost/app.xml . I have the feeling that it's something very simple, where only the last bit of information is missing in order to see the real problem.
And what do you recommend?
Define it in JNDI, but outside of your web application/WAR. In Tomcat, you can do this by putting it in the ${CATALINA_BASE}/conf/context.xml file. This allows external resource (mail, database etc) configuration to be defined outside of your web application, avoiding the need to repackage your WAR when you database configuration changes, or you move to a different environment with a different database, for instance.
I do the context look up in singleton Class called mailer.
This mailer class, is it in WEB-INF/classes, or a JAR within WEB-INF/lib? Or is it defined elsewhere in your classpath? If the latter, you may want to consider moving it into your application.
Edit: Based on your latest findings, it looks like your web app's META-INF/context.xml is not taking effect. There are several scenarios in Tomcat that will lead to this. I don't know these in detail, but here are some information I was able to find:
Per - http://tomcat.apache.org/tomcat-5.5-doc/config/host.html
If the "deployXML" attribute is set to false in your Host element (I believe in server.xml).
deployXML - Set to false if you want
to disable parsing the context.xml
file embedded inside the application
(located at /META-INF/context.xml).
Security consious environments should
set this to false to prevent
applications from interacting with the
container's configuration. The
administrator will then be responsible
for providing an external context
configuration file, and put it in
$CATALINA_HOME/conf/[enginename]/[hostname]/.
The flag's value defaults to true.
Per - http://tomcat.apache.org/tomcat-5.5-doc/config/context.html
If you have the $CATALINA_HOME/conf/[enginename]/[hostname]/[context-path].xml file defined.
I'm sure there are others, these are just the ones I was able to find quickly.
