Server FAILED_NOT_RESTARTABLE state in Oracle WebLogic Server 11g.

Today when I have go through my weblogic console I saw my one managed server in FAILED_NOT_RESTARTABLE state.
In my case after checking logs I find the following error message “ Unable to obtain lock on/u01/oid_domain/servers/wls_ods1/tmp/wls_ods1.lok. Server may already be running at internal.ServerLocks. getServerLock (”.

This could be because of two reasons
– WebLogic Server which you are trying to start is already running
– WebLogic Server which you are trying to start did not stop cleanly

Normally when weblogic server starts, it creates two lock files:
– $DOMAIN_HOME/servers//tmp/.lok
– $DOMAIN_HOME/servers//data/ldap/ldapfiles/EmbeddedLDAP.lok

When WebLogic server stops, it removes these two files. If you not stop server properly these lock files remains on that location and creating a problem when you attempt to start a server again and after more than one attempt your server goes in FAILED_NOT_RESTARTABLE state.

If you’ve got this issue, first identify if server is running on port configured for WebLogic Server by using netstat:

netstat -an | grep <WebLogic_Server_Port>

for Windows

netstat -aon | findstr <WebLogic_Server_Port>

If server is running then you should see output like this:

tcp        0      0 ::ffff:<IP>:<weblogic_port>      :::* LISTEN  

Note that there could be other process using this port (and that was my issue).

How to identify which port is configured for weblogic server?
To identify port configured for weblogic server open weblogic configuration file $DOMAIN_HOME/config/config.xml and search for listen-port, you should see entry like this:


If server is not running then you can safely remove these lok files under WebLogic server. And if server is running then trying to stop it from console, backend or by killing the process id of the particular server then remove the .lok files.

Important thing to know – if this is weblogic managed server (not Admin Server) then you can safely remove entire managed server directory (including sub directories) $DOMAIN_HOME/servers/ or rename it as a backup. When you start managed server again, Admin Server will create these directories automatically.