Apache Jackrabbit源码研究（一）

几年前某位大牛写了 深入浅出 jackrabbit 系列，链接地址为http://ahuaxuan.iteye.com/category/65829

本人读后受益匪浅（如果没用他的辅助之功，本人对jackrabbit的理解可能会摸索得更长），由于时隔久远，当时的jackrabbit版本为1.7，与现在的最新版本有点出入，本人抑制不住内心某种无名冲动，不顾自己理解上的肤浅，将自己对Apache Jackrabbit的源码解析记录下来，以期加深对编程的理解，或许有助于后来者

（注：本文目前可能还处于修改中，如需转载，害人害己）

jackrabbit对富文档的文本提取目前版本是通过apache tika实现的，这是与以前的版本不同的

实现该功能主要是LazyTextExtractorField类，该类继承自lucene的抽象类AbstractField

LazyTextExtractorField类的源码如下：

/**
* <code>LazyTextExtractorField</code> implements a Lucene field with a String
* value that is lazily initialized from a given {@link Reader}. In addition
* this class provides a method to find out whether the purpose of the reader
* is to extract text and whether the extraction process is already finished.
*
* @see #isExtractorFinished()
*/
public class LazyTextExtractorField extends AbstractField {

/**
* The logger instance for this class.
*/
private static final Logger log =
LoggerFactory.getLogger(LazyTextExtractorField.class);

/**
* The exception used to forcibly terminate the extraction process
* when the maximum field length is reached.
*/
private static final SAXException STOP =
new SAXException("max field length reached");

/**
* The extracted text content of the given binary value.
* Set to non-null when the text extraction task finishes.
*/
private volatile String extract = null;

/**
* Creates a new <code>LazyTextExtractorField</code> with the given
* <code>name</code>.
*
* @param name the name of the field.
* @param reader the reader where to obtain the string from.
* @param highlighting set to <code>true</code> to
*                     enable result highlighting support
*/
public LazyTextExtractorField(
Parser parser, InternalValue value, Metadata metadata,
Executor executor, boolean highlighting, int maxFieldLength) {
super(FieldNames.FULLTEXT,
highlighting ? Store.YES : Store.NO,
Field.Index.ANALYZED,
highlighting ? TermVector.WITH_OFFSETS : TermVector.NO);
executor.execute(
new ParsingTask(parser, value, metadata, maxFieldLength));
}

/**
* Returns the extracted text. This method blocks until the text
* extraction task has been completed.
*
* @return the string value of this field
*/
public synchronized String stringValue() {
try {
while (!isExtractorFinished()) {
wait();
}
return extract;
} catch (InterruptedException e) {
log.error("Text extraction thread was interrupted", e);
return "";
}
}

/**
* @return always <code>null</code>
*/
public Reader readerValue() {
return null;
}

/**
* @return always <code>null</code>
*/
public byte[] binaryValue() {
return null;
}

/**
* @return always <code>null</code>
*/
public TokenStream tokenStreamValue() {
return null;
}

/**
* Checks whether the text extraction task has finished.
*
* @return <code>true</code> if the extracted text is available
*/
public boolean isExtractorFinished() {
return extract != null;
}

private synchronized void setExtractedText(String value) {
extract = value;
notify();
}

/**
* Releases all resources associated with this field.
*/
public void dispose() {
// TODO: Cause the ContentHandler below to throw an exception
}

/**
* The background task for extracting text from a binary value.
*/
private class ParsingTask extends DefaultHandler implements Runnable {

private final Parser parser;

private final InternalValue value;

private final Metadata metadata;

private final int maxFieldLength;

private final StringBuilder builder = new StringBuilder();

public ParsingTask(
Parser parser, InternalValue value, Metadata metadata,
int maxFieldLength) {
this.parser = parser;
this.value = value;
this.metadata = metadata;
this.maxFieldLength = maxFieldLength;
}

public void run() {
try {
InputStream stream = value.getStream();
try {
parser.parse(stream, this, metadata, new ParseContext());
} finally {
stream.close();
}
} catch (Throwable t) {
if (t != STOP) {
log.warn("Failed to extract text from a binary property", t);
}
} finally {
value.discard();
}
setExtractedText(builder.toString());
}

@Override
public void characters(char[] ch, int start, int length)
throws SAXException {
builder.append(
ch, start,
Math.min(length, maxFieldLength - builder.length()));
if (builder.length() >= maxFieldLength) {
throw STOP;
}
}

@Override
public void ignorableWhitespace(char[] ch, int start, int length)
throws SAXException {
characters(ch, start, length);
}

}

}

从代码可以发现，富文档文本提取的工作是放在线程类ParsingTask中进行处理的，文本提取是通过异步方式进行的

这里的线程类同时继承自DefaultHandler，DefaultHandler实现了EntityResolver, DTDHandler, ContentHandler, ErrorHandler四接口，这是一种缺省适配器模式，为我们实现target目标接口提供便利

jaxp规范对xml格式文件的解析式基于事件监听模式，上面最主要的接口是ContentHandler，ParsingTask间接实现了该接口，同时将获取的文本增量累加在private final StringBuilder builder = new StringBuilder()对象里面

线程方法里面最后通过调用setExtractedText(builder.toString())方法提交得到的文本

需要注意的是，这里的parser对象，jackrabbit并没有使用原生的apache tika里面的类，而是封装了一个JackrabbitParser类

JackrabbitParser类的源码如下：

/**
* Jackrabbit wrapper for Tika parsers. Uses a Tika {@link AutoDetectParser}
* for all parsing requests, but sets it up with Jackrabbit-specific
* configuration and implements backwards compatibility support for old
* <code>textExtractorClasses</code> configurations.
*
* @since Apache Jackrabbit 2.0
*/
class JackrabbitParser implements Parser {

/**
* Logger instance.
*/
private static final Logger logger =
LoggerFactory.getLogger(JackrabbitParser.class);

/**
* Flag for blocking all text extraction. Used by the Jackrabbit test suite.
*/
private static volatile boolean blocked = false;

/**
* The configured Tika parser.
*/
private final AutoDetectParser parser;

/**
* Creates a parser using the default Jackrabbit-specific configuration
* settings.
*/
public JackrabbitParser() {
InputStream stream =
JackrabbitParser.class.getResourceAsStream("tika-config.xml");
try {
if (stream != null) {
try {
parser = new AutoDetectParser(new TikaConfig(stream));
} finally {
stream.close();
}
} else {
parser = new AutoDetectParser();
}
} catch (Exception e) {
// Should never happen
throw new RuntimeException(
"Unable to load embedded Tika configuration", e);
}
}

/**
* Backwards compatibility method to support old Jackrabbit 1.x
* <code>textExtractorClasses</code> configurations. Implements a best
* effort mapping from the old-style text extractor classes to
* corresponding Tika parsers.
*
* @param classes configured list of text extractor classes
*/
public void setTextFilterClasses(String classes) {
Map<MediaType, Parser> parsers = new HashMap<MediaType, Parser>();

StringTokenizer tokenizer = new StringTokenizer(classes, ", \t\n\r\f");
while (tokenizer.hasMoreTokens()) {
String name = tokenizer.nextToken();
if (name.equals(
"org.apache.jackrabbit.extractor.HTMLTextExtractor")) {
parsers.put(MediaType.text("html"), new HtmlParser());
} else if (name.equals("org.apache.jackrabbit.extractor.MsExcelTextExtractor")) {
Parser parser = new OfficeParser();
parsers.put(MediaType.application("vnd.ms-excel"), parser);
parsers.put(MediaType.application("msexcel"), parser);
parsers.put(MediaType.application("excel"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.MsOutlookTextExtractor")) {
parsers.put(MediaType.application("vnd.ms-outlook"), new OfficeParser());
} else if (name.equals("org.apache.jackrabbit.extractor.MsPowerPointExtractor")
|| name.equals("org.apache.jackrabbit.extractor.MsPowerPointTextExtractor")) {
Parser parser = new OfficeParser();
parsers.put(MediaType.application("vnd.ms-powerpoint"), parser);
parsers.put(MediaType.application("mspowerpoint"), parser);
parsers.put(MediaType.application("powerpoint"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.MsWordTextExtractor")) {
Parser parser = new OfficeParser();
parsers.put(MediaType.application("vnd.ms-word"), parser);
parsers.put(MediaType.application("msword"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.MsTextExtractor")) {
Parser parser = new OfficeParser();
parsers.put(MediaType.application("vnd.ms-word"), parser);
parsers.put(MediaType.application("msword"), parser);
parsers.put(MediaType.application("vnd.ms-powerpoint"), parser);
parsers.put(MediaType.application("mspowerpoint"), parser);
parsers.put(MediaType.application("vnd.ms-excel"), parser);
parsers.put(MediaType.application("vnd.openxmlformats-officedocument.wordprocessingml.document"), parser);
parsers.put(MediaType.application("vnd.openxmlformats-officedocument.presentationml.presentation"), parser);
parsers.put(MediaType.application("vnd.openxmlformats-officedocument.spreadsheetml.sheet"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.OpenOfficeTextExtractor")) {
Parser parser = new OpenDocumentParser();
parsers.put(MediaType.application("vnd.oasis.opendocument.database"), parser);
parsers.put(MediaType.application("vnd.oasis.opendocument.formula"), parser);
parsers.put(MediaType.application("vnd.oasis.opendocument.graphics"), parser);
parsers.put(MediaType.application("vnd.oasis.opendocument.presentation"), parser);
parsers.put(MediaType.application("vnd.oasis.opendocument.spreadsheet"), parser);
parsers.put(MediaType.application("vnd.oasis.opendocument.text"), parser);
parsers.put(MediaType.application("vnd.sun.xml.calc"), parser);
parsers.put(MediaType.application("vnd.sun.xml.draw"), parser);
parsers.put(MediaType.application("vnd.sun.xml.impress"), parser);
parsers.put(MediaType.application("vnd.sun.xml.writer"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.PdfTextExtractor")) {
parsers.put(MediaType.application("pdf"), new PDFParser());
} else if (name.equals("org.apache.jackrabbit.extractor.PlainTextExtractor")) {
parsers.put(MediaType.TEXT_PLAIN, new TXTParser());
} else if (name.equals("org.apache.jackrabbit.extractor.PngTextExtractor")) {
Parser parser = new ImageParser();
parsers.put(MediaType.image("png"), parser);
parsers.put(MediaType.image("apng"), parser);
parsers.put(MediaType.image("mng"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.RTFTextExtractor")) {
Parser parser = new RTFParser();
parsers.put(MediaType.application("rtf"), parser);
parsers.put(MediaType.text("rtf"), parser);
} else if (name.equals("org.apache.jackrabbit.extractor.XMLTextExtractor")) {
Parser parser = new XMLParser();
parsers.put(MediaType.APPLICATION_XML, parser);
parsers.put(MediaType.text("xml"), parser);
} else {
logger.warn("Ignoring unknown text extractor class: {}", name);
}
}

parser.setParsers(parsers);
}

/**
* Delegates the call to the configured {@link AutoDetectParser}.
*/
public Set<MediaType> getSupportedTypes(ParseContext context) {
return parser.getSupportedTypes(context);
}

/**
* Delegates the call to the configured {@link AutoDetectParser}.
*/
public void parse(
InputStream stream, ContentHandler handler,
Metadata metadata, ParseContext context)
throws IOException, SAXException, TikaException {
waitIfBlocked();
parser.parse(stream, handler, metadata, context);
}

public void parse(
InputStream stream, ContentHandler handler, Metadata metadata)
throws IOException, SAXException, TikaException {
parse(stream, handler, metadata, new ParseContext());
}

/**
* Waits until text extraction is no longer blocked. The block is only
* ever activated in the Jackrabbit test suite when testing delayed
* text extraction.
*
* @throws TikaException if the block was interrupted
*/
private synchronized static void waitIfBlocked() throws TikaException {
try {
while (blocked) {
JackrabbitParser.class.wait();
}
} catch (InterruptedException e) {
throw new TikaException("Text extraction block interrupted", e);
}
}

/**
* Blocks all text extraction tasks.
*/
static synchronized void block() {
blocked = true;
}

/**
* Unblocks all text extraction tasks.
*/
static synchronized void unblock() {
blocked = false;
JackrabbitParser.class.notifyAll();
}

}

具体的文本解析工作是通过委托给AutoDetectParser类来执行的，如果看过我以前的apache tika源码研究，就可以知道AutoDetectParser类继承自CompositeParser类，而CompositeParser类的处理方式是通过调用它的Parser聚集来完成具体的解析工作，这里面 实现的是composite模式（自顶向下的安全式的composite模式）

---------------------------------------------------------------------------

本系列Apache Jackrabbit源码研究系本人原创

转载请注明出处 博客园 刺猬的温驯

本文链接 http://www.cnblogs.com/chenying99/archive/2013/04/03/2997156.html