Thursday, February 28, 2013

Start and close Apache Solr by Java methods instead of command lines


The Apache Solr server on the local computer can be started by running a java -jar start.jar command in Cygwin or cmd and closed by a Ctrl + C command. However, it's not convenient to manually check whether the Solr server is started, and an additional Cygwin or cmd window on the taskbar would be quite annoying. It's obviously a better design to do these via a few Java modules.

This post is a tutorial on how to start Solr server, close Solr server, and check whether the local Solr server is running or not by Java. I searched on the Internet, read some StackOverFlow posts, and wrote the code below. The three methods startSolrServer(), closeSolrServer(), and isSolrServerAlive() are responsible for the three functionalities respectively.

You can simply copy the three Java classes SyncPipe.java, LoggedPrintStream.java, and SolrUtilities.java below, paste them at some whether in your Java project, configure the solrDirectory, and directly use the three methods.

Beside, I feel that the design of the closeSolrServer() method is a bit inelegant. Weclome to leave comments below to provide better solutions about closing the Solr server by Java.



 

SyncPipe.java

package com.solrutils;

import java.io.InputStream;
import java.io.OutputStream;

public class SyncPipe implements Runnable {
 public SyncPipe(InputStream istrm, OutputStream ostrm) {
  istrm_ = istrm;
  ostrm_ = ostrm;
 }

 public void run() {
  try {
   final byte[] buffer = new byte[1024];
   for (int length = 0; (length = istrm_.read(buffer)) != -1;) {
    ostrm_.write(buffer, 0, length);
   }
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 private final OutputStream ostrm_;
 private final InputStream istrm_;
}


LoggedPrintStream.java

package com.solrutils;

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintStream;
import java.lang.reflect.Field;

public class LoggedPrintStream extends PrintStream {

 final StringBuilder buf;
 final PrintStream underlying;

 LoggedPrintStream(StringBuilder sb, OutputStream os, PrintStream ul) {
  super(os);
  this.buf = sb;
  this.underlying = ul;
 }

 public static LoggedPrintStream create(PrintStream toLog) {
  try {
   final StringBuilder sb = new StringBuilder();
   Field f = FilterOutputStream.class.getDeclaredField("out");
   f.setAccessible(true);
   OutputStream psout = (OutputStream) f.get(toLog);
   return new LoggedPrintStream(sb, new FilterOutputStream(psout) {
    public void write(int b) throws IOException {
     super.write(b);
     sb.append((char) b);
    }
   }, toLog);
  } catch (NoSuchFieldException shouldNotHappen) {
  } catch (IllegalArgumentException shouldNotHappen) {
  } catch (IllegalAccessException shouldNotHappen) {
  }
  return null;
 }
}

SolrUtilities.java 

package com.solrutils;

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;

public class SolrUtilities {
 private static final String solrDirectory = "D:/solr-4.1.0/";

 public static void startSolrServer() {
  try {
   String[] command = { "cmd", };
   Process p = Runtime.getRuntime().exec(command);
   new Thread(new SyncPipe(p.getErrorStream(), System.err)).start();
   new Thread(new SyncPipe(p.getInputStream(), System.out)).start();
   PrintWriter stdin = new PrintWriter(p.getOutputStream());
   stdin.println("cd /d D:");
   stdin.println("cd " + solrDirectory + "example");
   stdin.println("java -jar start.jar");
   stdin.close();

  } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
 }

 public static boolean isSolrServerAlive() {

  try {
   LoggedPrintStream lpsOut = LoggedPrintStream.create(System.out);
   LoggedPrintStream lpsErr = LoggedPrintStream.create(System.err);

   System.setOut(lpsOut);
   System.setErr(lpsErr);

   String[] command = { "cmd", };
   Process p = Runtime.getRuntime().exec(command);
   new Thread(new SyncPipe(p.getInputStream(), System.out)).start();
   new Thread(new SyncPipe(p.getInputStream(), System.out)).start();
   PrintWriter stdin = new PrintWriter(p.getOutputStream());
   stdin.println("netstat -o -n -a | findstr 0:8983");
   stdin.flush();
   while (lpsOut.buf.lastIndexOf("findstr") == -1) {
    System.out.println("lpsOut.buf = " + lpsOut.buf);
    try {
     Thread.sleep(250);
    } catch (InterruptedException e) {
     e.printStackTrace();
    }
   }
   System.setOut(lpsOut.underlying);
   stdin.close();
   if (lpsOut.buf.indexOf("LISTENING") == -1) {
    return false;
   } else {
    return true;
   }

  } catch (Exception e) {
   e.printStackTrace();
   return false;
  }

 }

 public static void closeSolrServer() {

  try {
   LoggedPrintStream lpsOut = LoggedPrintStream.create(System.out);
   System.setOut(lpsOut);

   String[] command = { "cmd", };
   Process p = Runtime.getRuntime().exec(command);
   new Thread(new SyncPipe(p.getErrorStream(), System.err)).start();
   new Thread(new SyncPipe(p.getInputStream(), System.out)).start();
   OutputStream outputStream = p.getOutputStream();
   PrintWriter stdin = new PrintWriter(outputStream);
   stdin.println("netstat -o -n -a | findstr 0:8983");
   stdin.flush();
   while (lpsOut.buf.lastIndexOf("findstr") == -1) {
    try {
     Thread.sleep(200);
    } catch (InterruptedException e) {
     e.printStackTrace();
    }
   }
   System.setOut(lpsOut.underlying);

   if (lpsOut.buf.indexOf("LISTENING") == -1) {
    System.err.println("SolrUtilities 62 the Solr server is already closed");
    return;
   } else {
    String thePIDString = lpsOut.buf.substring(
      lpsOut.buf.indexOf("LISTENING") + 9).trim();
    int i = 0;
    while (0 <= thePIDString.charAt(i) - '0'
      && thePIDString.charAt(i) - '0' <= 9)
     i++;
    thePIDString = thePIDString.substring(0, i);
    stdin.println("taskkill /f /pid " + thePIDString);
   }
   stdin.close();

  } catch (Exception e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }

 }
 
 public static void main(String[] args) {

  boolean isSolrAlive = isSolrServerAlive();
  System.out.println("SolrUtilities.main() 122 isSolrServerAlive? " + isSolrAlive);
  
  if (!isSolrAlive) {
   startSolrServer();
   System.out.println("SolrUtilities.main() the Solr server is started");
  }
  
  try {
   Thread.sleep(30000);
  } catch (InterruptedException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
  
  closeSolrServer();
  System.out.println("SolrUtilities.main() the Solr server is closed");
  
 }
}

Step-by-step tutorial on the installation of Apache Solr


This post is a simple, detailed, and step-by-step tutorial on how to install Apache Solr


Step 1: Download solr-4.1.0 from http://www.apache.org/dist/lucene/solr/4.1.0/solr-4.1.0.zip

Step 2: Unzip solr-4.1.0 to a folder. Suppose the path of the folder is D:/solr-4.1.0

Step 3: Execute cmd.exe

Step 4: Enter command: cd /d D:  to change disk

Step 5: Enter command: cd D:/solr-4.1.0/example/

This step is to change to the directory of the start.jar.

Step 6: Enter command: java -jar start.jar to start Apache Solr.

If the resulting screen is without error messages like this, it is successful.



After entering this command, do not shutdown the cmd window or enter Ctrl C to close the server now.

Step 7: Execute cmd.exe again to open another cmd window

Step 8: In the new cmd window, enter command: cd /d D:

Step 9: In the same cmd window, enter cd D:/solr-4.1.0/example/exampledocs

This step is to change to the directory of the post.jar

Step 10: In the same window, enter java -Dauto -Drecursive -jar post.jar D:/solr-4.1.0/example/exampledocs

This step is to index all Solr documents inside the folder D:/solr-4.1.0/example/exampledocs
with the post.jar. If the resulting screen is similar to this, this step is successful.

 


After this step is done, the Solr documents inside D:/solr-4.1.0/example/exampledocs are successfully indexed. Then you can quickly search the indexed documents with Solr queries.

For example, you can search documents with "George" in the author field and "game" in the game field by this url
http://localhost:8983/solr/collection1/select?q=author:George%20AND%20name:game&wt=xml and get the following Solr document:




-------



P.S: Some commands for Solr can be looked up by "java -jar post.jar -h." The post.jar is inside D:/solr-4.1.0/example/exampledocs. The following are example commands:

  java -jar post.jar *.xml

  java -Ddata=args  -jar post.jar '<delete><id>42</id></delete>'

  java -Ddata=stdin -jar post.jar < hd.xml

  java -Ddata=web -jar post.jar http://example.com/

  java -Dtype=text/csv -jar post.jar *.csv

  java -Dtype=application/json -jar post.jar *.json

  java -Durl=http://localhost:8983/solr/update/extract -Dparams=literal.id=a -Dt                   ype=application/pdf -jar post.jar a.pdf

  java -Dauto -jar post.jar *

  java -Dauto -Drecursive -jar post.jar afolder

Furthermore, you can install Cygwin and the curl package and then index documents with a curl command:
curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary @books.json -H 'Content-type:application/json'