Tinkering the Java classpath to fix ClassNotFoundException
So I recently got an annoying ClassNotFoundException. I browsed the net and I found the solution here at IBM website by Elliotte Rusty Harold. I copy here the article, just in case the original article disappear.
Figure 1. Directory structure follows the package structure
Figure 2. Parallel source and compiled hierarchies
The sourcepath
Listing 1. A class in one package can import a class in another
Figure 3. Source structure for several packages
Now let's see what happens when I try to compile MainFrame.java like I did before:
Listing 2. Compiling MainFrame.java
Figure 4. Multiclass output
Compiling multiple directories in the sourcepath
Of course, you can use various forms of relative paths if you prefer. For instance, if project1 and project2 are siblings of the current working directory (that is, they have the same parent directory), then I could reference them like this:
Figure 5. Quick fix for the classpath in Eclipse
Keep in mind, however, that these IDEs still sit on top of a filesystem that must be set up properly, especially if you need to integrate with other tools and other IDEs. The main contribution of these tools is that GUI dialogs, tree views, and tabs replace command-line switches; but the basic file structure is the same.
The classpath is the connection between the Java runtime and the filesystem. It defines where the interpreter looks for .class files to load. The basic idea is that the filesystem hierarchy mirrors the Java package hierarchy, and the classpath specifies which directories in the filesystem serve as roots for the Java package hierarchy.
Unfortunately, file systems are complex and very platform dependent, and they don't perfectly match Java packages. Consequently, the classpath has been a thorn in the side of both new users and experienced Java programmers for years. It isn't the pretty part of the Java platform. It is the annoying glitch that keeps you working well past 5 p.m., trying to debug a small problem that stubbornly refuses solution.
A good IDE like Eclipse can shield you from some of the difficulties of managing the classpath, but only somewhat, and only as long as nothing goes wrong (and something always goes wrong). Consequently, it is essential that every Java programmer fully understand the classpath. Only with in-depth understanding can you hope to debug the thorny problems that arise from the classpath.
In this article, I lay out everything you need to know about the Java classpath (and the associated sourcepath) on UNIX, Linux, and Mac OS X. In the companion article, I demonstrate similar techniques for Windows. Following the procedures outlined here will serve as a guide along the way, and should cure most classpath problems.
Package structure
Mastering the classpath begins with the source code. Every class belongs in a package, and this package must follow the standard naming conventions. To briefly review: A package name begins with a two-level reversed domain name such as
com.example or edu.poly. This is followed by at least one more word that describes the contents of the package. For example, because I own the domain name elharo.com, if I were to write a Fraction class, I might place it in one of the following packages:com.elharo.mathcom.elharo.numberscom.elharo.math.algebra.fields
After the reversed domain name, use only single-word subpackage names. Do not abbreviate, and do spell all words correctly. Use a spell checker if you need to. A large percentage of classpath-related problems are caused by using one word in the source code and a slightly different spelling or abbreviation of that word in the filesystem. The only sensible choice is to always use correctly spelled, unabbreviated names.
The entire package name should be lowercase, even for proper names and acronyms that are normally capitalized. The package name should be composed exclusively of ASCII characters. While the compiler accepts package names written in Hebrew, Cyrillic, Greek, and other scripts, many file systems don't. As you'll see shortly, these package names will have to serve double duty as directory names. Consequently, package (and class) names should be limited to ASCII. (While Java package and class names are Unicode, many file systems are not yet Unicode-savvy. Simply copying a file to a system with a different default encoding can keep the compiler and interpreter from finding the right classes.)
Do not skimp on your package name! It will only lead to disaster in the long run. If you need a domain name, buy one. If the names are too long, buy a shorter one. (I once bought xom.nu so my package prefix was only six characters long.) Do not place your classes in the default package (the package you get if you don't include a package statement in the class). If package access prevents objects from communicating, add more public methods to the classes. Every class you use more than once must be in a package.
Directory structure
The next step is to organize your source files to match the package structure. Create a clean, empty directory somewhere. For the purposes of this article, I'll name it project. Inside this directory, create two more directories: bin and src. (Some people prefer to name these build and source, respectively.)
Next, inside the src directory, make a hierarchy that mirrors your package hierarchy. For example, given a class named
com.elharo.math.Fraction, I would place a com directory in the src directory. Then I would create an elharo directory inside the com directory. Then I would put a math directory inside the elharo directory. Finally, I would put Fraction.java inside this math directory, as shown in Figure 1:Figure 1. Directory structure follows the package structure
You can do this with one
mkdir -p command:$ mkdir -p com/elharo/math |
Very important: Never put anything other than source code in your src directory. Usually the only files you put there are .java files. On occasion, you may place .html files (for Javadoc) or other types of source code in this directory. However, you never want to put .class files or other compiled, generated artifacts in this hierarchy. Doing so is a recipe for disaster. Sadly, the javac compiler will do exactly that unless you're careful. In the next section, I'll show you how to fix that.
Compiling
Compiling Java code is tricky because you need to keep track of several related but different things:
- The target file you're compiling.
- The directory where the compiler looks for .java files that the target file imports.
- The directory where the compiler looks for .class files the target file imports.
- The directory where the compiler puts the compiled output.
By default, the javac compiler thinks all of these are the current working directory, which is almost never what you want. Consequently, you need to explicitly specify each of these elements when you compile.
The file to compile
The first thing you specify is the .java file you're going to compile. This is given as a path to that file from the current working directory. For example, suppose you are in the project directory shown in Figure 1. This directory contains an src directory. The src directory contains a com directory, which contains the example directory, which contains the Fraction.java file. The following command line compiles it:
$ javac src/com/elharo/math/Fraction.java |
If the path is incorrect, you'll get an error message like this one:
error: cannot read: src/com/example/mtah/Fraction.java |
If you see this error message, check each piece of the path to make sure it's spelled correctly. Then check that the file really is where it's supposed to be by doing an
ls like the one shown here:$ ls src/com/example/math |
This problem usually indicates a mistyped path, but it can also mean that you're not in the directory you think you're in. In this example, you'd check to see that the current working directory is the project directory. The
pwd command is helpful here. For example, the following tells me that I'm actually in the project/src instead of the project directory:$ pwd |
I need to
cd .. before compiling.Where the output goes
Assuming there are no syntax errors, javac places the compiled .class file in the same directory where the .java file is. You do not want this. Mixing .class and .java files makes it very hard to clean up the compiled files without accidentally deleting .java files you want to keep. This makes clean builds problematic and tends to result in versioning problems. It also makes it hard to jar up just the compiled .class files when distributing a binary. Therefore, you need to tell the compiler to put the compiled output in a completely different directory. The
-d switch specifies the output directory (usually called bin, build, or classes):$ javac -d bin src/com/elharo/math/Fraction.java |
Now the output is as shown in Figure 2. Notice that javac has created the complete com/elharo/math hierarchy of directories. You do not need to do it manually.
Figure 2. Parallel source and compiled hierarchies
The sourcepath
The directory where Java looks for source files is called the sourcepath. In the scheme outlined here, this is the src directory. It is the directory that contains the hierarchy of source files, organized into their own directories. It is not the com directory or the src/com/elharo/math directory.
Most projects use more than one class and more than one package. These are connected by import statements and fully package-qualified class names. For example, suppose you now create a new
MainFrame class in the com.elharo.guipackage, as shown in Listing 1:Listing 1. A class in one package can import a class in another
package com.elharo.gui; |
This class uses the
com.elharo.math.Fraction class in a different package from the MainFrame class. The source setup is now as shown in Figure 3. (I have deleted the compiled output from the previous step. I can always compile it again.)Figure 3. Source structure for several packages
Now let's see what happens when I try to compile MainFrame.java like I did before:
Listing 2. Compiling MainFrame.java
$ javac -d bin src/com/elharo/gui/MainFrame.java |
The errors in Listing 2 happened because although javac knew where to find MainFrame.java, it didn't know where to find Fraction.java. (You'd think it would be smart enough to notice the matching package hierarchies, but it's not.) To clue it in, I have to specify the sourcepath. This specifies the directories where the compiler looks for the hierarchy of source files. In Listing 2, that's src. So I use the
-sourcepath option, like so:$ javac -d bin -sourcepath src src/com/elharo/gui/MainFrame.java |
Now the program compiles without error and produces the output shown in Figure 4. Notice that javac also compiled the file Fraction.java, referenced by the file I was compiling.
Figure 4. Multiclass output
Compiling multiple directories in the sourcepath
You can actually have more than one directory in your sourcepath, separated by colons, though this is usually not necessary. For example, if I want to include both the local src directory and the directory /Users/elharo/Projects/XOM/src where I keep the source code for another project, I can compile like this:
$ javac -d bin -sourcepath src:/Users/elharo/Projects/XOM/src |
This command does not compile every file found in either of those hierarchies. It only compiles the files referenced directly or indirectly by the single .java file I explicitly ask to be compiled.
Much more commonly, you'll have a single source directory for .java files, but multiple directories for classes or JAR archives where precompiled third party libraries are placed. This is the role of the classpath.
Setting the classpath
In medium-to-large projects, recompiling every file every time can be time-consuming. You can ease this burden by separately compiling and storing independent parts of the same project in different classes or bin directories. These directories are added to the classpath.
There are several ways to add a class to the classpath. The
-classpath command-line switch is the only one you should use, however. For example, suppose I want to import files from another project that I've previously compiled into the directory /Users/elharo/classes. Then I would add -classpath /Users/elharo/classes to the command line like so:$ javac -d bin -sourcepath src -classpath /Users/elharo/classes |
Now suppose I need to add two directories, /Users/elharo/project1/classes and /Users/elharo/project2/classes. Then I would include them both separated by a colon, like this:
$ javac -d bin -sourcepath src |
Of course, you can use various forms of relative paths if you prefer. For instance, if project1 and project2 are siblings of the current working directory (that is, they have the same parent directory), then I could reference them like this:
$ javac -d bin -sourcepath src |
So far, I've assumed the program is complete onto itself and does not use any separately compiled third-party libraries. If it does, you'll need to add them to the classpath too. Libraries are usually distributed as JAR files such as junit.jar or icu4j.jar. In this case, it is the JAR file itself that you add to the classpath, not the directory that contains it. (In essence, the JAR file acts as a directory that contains compiled .class files.) For example, the following command adds three things to the classpath: the directory /Users/elharo/classes, the file icu4j.jar in the current working directory, and the file junit.jar in /Users/elharo/lib:
$ javac -d bin -sourcepath src |
JAR files are only used for .class files and the classpath, not for .java files and the sourcepath.
Running the program
You have now successfully compiled your program, and are ready to run it. This is similar to but simpler than compiling. When running a program, you only need to specify two things:
- The classpath.
- The fully package qualified name of the class that contains your
main()method.
You do not need to specify the sourcepath.
Usually the classpath is the same classpath you used to compile the program, with the addition of the directory where the compiled output was placed. For example, if the compile command was this:
$ javac -d bin -sourcepath src |
and the
main() method was in the class com.elharo.gui.MainFrame, then you would run the program like this:$ java |
Take careful note that the last item on the command line is a class name. It is not a file name. It does not end in .java or .class. This class must be found somewhere on the classpath.
Other places classes reside
I strongly recommend that you always explicitly specify the classpath when you compile and when you run. There are other places you can put files so that they are added to the classpath and found by both the javac compiler and the java interpreter. These options save only a little amount of typing, and they do so at the expense of a large amount of debugging when -- not if -- you accidentally place an old version of a class in the classpath.
In this section, I'll show you some of the places you might expect to find classes hiding, that unexpectedly pop into your classpath and cause problems. This is especially likely to happen on machines you don't control, such as a server.
The current working directory
The compiler uses the current working directory (.) as the default classpath. However, as soon as you set the classpath in some other way--e.g. with
-classpath or the CLASSPATH environment variable--this is no longer automatic. You have to add the current working directory to the classpath just like any other directory. In either case, it's too easy to forget what is or isn't in the same directory as you are. Thus, try to avoid putting any classes or hierarchies into your project or home directory. Instead, always keep things neatly separated into src directories for .java files and bin directories for .class files.CLASSPATH
After a while, you may get tired of manually adding the bin directories and JAR archives to the classpath. You may then discover the
CLASSPATH environment variable. You can add directories and JAR archives just once to the CLASSPATH environment variable. Then you don't have to type their paths every time you run javac or java.Resist this temptation. It will cause problems when you load the wrong class or the wrong version of a class. Any time you save typing now will be taken back from you a hundred times over in debugging problems caused by accidentally loading the wrong classes. There are better ways to automate classpaths and avoid typing.
jre/lib/ext
JAR archives placed in your jre/lib/ext directory are added to the classpath of all applications run with that virtual machine. While this seems convenient, it is also a long-term mistake akin to adding directories to the
CLASSPATH environment variable. Sooner or later (probably sooner), you'll load the wrong version of a class from a place you aren't even thinking about and waste hours debugging.This problem is especially serious when deploying server-side applications. Be very careful that the server you're deploying to doesn't have any extra JARs in its jre/lib/ext directory. Problems caused by the wrong version of a JAR archive in the classpath can be extremely hard to debug if you don't recognize the symptoms or know just what to look for. To avoid these problems, some frameworks have even gone so far as to write their own class loaders that bypass Java code's usual class loading mechanisms.
jre/lib/endorsed
JAR files in the jre/lib/endorsed directory are also added to the classpath of all applications run with that virtual machine. The difference is that these files are actually added to the bootclasspath rather than the usual classpath and can replace the standard classes shipped with the JDK. This approach is especially useful for upgrading the XML parser and fixing bugs in the VM.
Once again, though, while this technique seems convenient, it is also a long-term mistake for the same reason. If you need to replace JDK classes, use the
-Xbootclasspath/p option at runtime to avoid accidentally loading the wrong version of a class:$ java -classpath /Users/elharo/classes |
Automating classpath management
You should learn to use a hammer before you pick up a nail gun. Likewise, you should become comfortable managing classes manually before you try to use more powerful tools. However, there are tools designed to alleviate the pain of dealing with the sourcepath and classpath. Mostly they do this by organizing files for you along the lines I've laid out in this article.
IDEs
Most integrated development environments such as Eclipse and NetBeans automate and assist with some aspects of classpath management. For instance, when you change a package name, Eclipse offers to move the corresponding .java file to match, as shown in Figure 5:
Figure 5. Quick fix for the classpath in Eclipse
Keep in mind, however, that these IDEs still sit on top of a filesystem that must be set up properly, especially if you need to integrate with other tools and other IDEs. The main contribution of these tools is that GUI dialogs, tree views, and tabs replace command-line switches; but the basic file structure is the same.
Ant
Ant is the de facto standard tool for automating the build process. Unlike putting directories in jre/lib/ext or the
CLASSPATHenvironment variable, Ant really does let you create one-step build processes. You still need to set up the classpath in your Ant build.xml file and manually put the source files in the right directories, but at least you don't need to keep respecifying it every time you compile.Maven
Maven goes even further than Ant in organizing and automating the build process and associated classpath issues. Maven provides a reasonable default setup that enables you to build simple projects with just a few lines of code, as long as you put the source files where Maven expects to find them. You still have to coordinate the filesystem hierarchy and the package hierarchy. Maven is especially good at managing dependencies on third-party libraries, though it's not as easily customizable as Ant.
In conclusion
As troublesome as the classpath is, you can tame it with a few simple rules. In particular:
- Place every class in a package.
- Rigorously follow package and class naming and capitalization conventions.
- Make sure your package hierarchy matches the directory hierarchy.
- Always use the
-doption to javac. - Never put anything in jre/lib/ext.
- Never put anything in jre/lib/endorsed.
- Never put .java files in the same directory as .class files.
- Never put any .java or .class files in the current working directory.
One final tip: A lot of time-killing problems with the classpath revolve around simple errors like misspelling a directory name or compiling from the wrong directory. If you just can't figure out what's going wrong, ask a friend or colleague to look at your problem. Often I find I'm too close to the problem to see a bug in my setup that's immediately obvious to anyone else. A second pair of eyes is a very effective debugging technique.
The classpath is certainly not easy, but there is a method to its madness and it is manageable. A bit of care and some attention paid to naming conventions, command-line arguments, and directory structures should enable you to compile and run programs with a minimum of fuss.
Resources
Learn
- "Managing the Java classpath on Windows" (Elliotte Harold, developerWorks, December 2006): The companion article to this one describes how to set up the classpath and sourcepath on Windows.
- "IBM Cloudscape: Understanding the Java classpath" (Jean Anderson and Susan Cline, developerWorks, September 2004): Explains how to set up the Java classpath for Cloudscape and Derby.
- "Best Practice: Classpath structure for WebSphere Application Server" (WebSphere Best Practices team, developerWorks, August 2001): Recommendations for managing the classpath on WebSphere Application Server.
- developerWorks technical library for Java: Find a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks related to the Java platform.