This page last changed on Oct 08, 2008 by tonyj.

How to Fix Confluence/JIRA

Owned by: Tony Johnson 

Confluence is a server-based, knowledge management tool enabling people to access, create, and edit content for web pages using a browser and the wiki markup language. Test (and fix) in the following order:  

How to Fix:
  1. Web Server
  2. tomcat Server
  3. MySQL
  4. tomcat connector
  5. Other Dependencies

Test and Fix

 1. Web Server

To Test
  • Enter (in your web browser): http://web007.slac.stanford.edu
    • The Web Server is working if you get a response beginning with: Web007.slac.stanford.edu ...

  • To Fix: Call Windows xHELP at (650) 926-4357.  You will get the Help menu.

    Select "page the on-call Technical Coordinator" who will try to assist, or will contact a member of the Windows Infrastructure Group as needed.
  • Alternatively contact Richard, Max or Tony who can log in to web007.slac.stanford.edu:1023 using windows remote desktop and can restart the IIS server using the internet services manager_._

    Note: If confluence has been down for some time it can cause web007 to hang, so if after restarting web007 confluence still does not work in may be necessary to also restart the tomcat server as detailed below.

2. tomcat Server  

To Test
  • Using the ServerMonitoring tool, check the status tomcat04.

  • To Fix:  Note that  tomcat servers are normally started automatically when the machine boots via cron jobs running under the glast account. Each server runs under control of the java service wrapper, which will automatically restart the server if it fails.

    If it is necessary to do a manual restart, log in to the failed machine as user "glast"; then issue the command:  ~glast/tomcat/scripts/tomcat restart

3. MySQL

To Test
  • Check: Nagios MySQL Service Group
    • Note that the MySQL Master and slave are identified in the Service column.
    • Tip: For additional status information on the Master and Slave, click on their respective stoplight icons in the Actions column.

  • To Fix: (root permission is required to perform the following procedures)
    Note that the most common problem is that u02 on the master server fills up.
    To Fix u02 :
    1. Login to the slave mysql server as: root
    2. At the prompt enter: mysql 
    3. At the prompt, enter: show slave status   A table with an entry of what the latest binary file the slave is reading is (e.g., mysql-bin.XXXXXX) will be displayed. 
    4. Go to the master mysql server (glastlnx01) and login as: root 
    5. At the prompt, enter: purge master logs to mysql-bin.XXXXXX-1   where: XXXXXX is the same as from the slave in the previous command decremented by 1.

  • To Fix All Other Problems: All other problems can usually be fixed by restarting the master mysql server (glastlnx01).
    1. Login to the master mysql server (glastlnx01) as: root
    2. At the command line, enter: /etc/init.d/mysql restart

5. Other Dependencies

Confluence can also fail if:

  • the glast unix account home directory disappears, or (currently)
  • if there are problems with u19 or sulky33 (fileserver that hosts u19)

       Check:  [Nagios|http://glast-nagios.slac.stanford.edu/nagios/] ? Service Detail


Document generated by Confluence on Jan 21, 2010 11:37