Lucene根据字段进行自定义搜索扩展

最近需要对公司的产品搜索功能做一步改动，搜索到的结果首先按照是否有库存进行排序，然后再按照销量。由于库存量也是一个整数，如果直接按照库存量进行倒序排序的话，是不符合要求的，Lucene也没有支持我们这种特殊的业务需求，但是可以通过扩展的方式进行改写。
 
参考文档：http://blog.csdn.net/cctcc/article/details/45672247

 

public class EmptyStockComparatorSource extends FieldComparatorSource {
@Override
public FieldComparator<?> newComparator(String fieldname, int numHits, int sortPos, boolean reversed)
throws IOException {
return new LongComparator(numHits, fieldname, 0L);
}

public static class LongComparator extends FieldComparator.NumericComparator<Long> {
private final long[] values;
private long bottom;
private long topValue;

/**
* Creates a new comparator based on {@link Long#compare} for {@code numHits}.
* When a document has no value for the field, {@code missingValue} is substituted.
*/
public LongComparator(int numHits, String field, Long missingValue) {
super(field, missingValue);
values = new long[numHits];
}

@Override
protected void doSetNextReader(LeafReaderContext context) throws IOException {
currentReaderValues = getNumericDocValues(context, field);
if (missingValue != null) {
docsWithField = getDocsWithValue(context, field);
// optimization to remove unneeded checks on the bit interface:
if (docsWithField instanceof Bits.MatchAllBits) {
docsWithField = null;
}
} else {
docsWithField = null;
}
}

@Override
public int compare(int slot1, int slot2) {
return Long.compare(values[slot1], values[slot2]);
}

@Override
public int compareBottom(int doc) {
// TODO: there are sneaky non-branch ways to compute
// -1/+1/0 sign
long v2 = currentReaderValues.get(doc);
// Test for v2 == 0 to save Bits.get method call for
// the common case (doc has value and value is non-zero):
if (docsWithField != null && v2 == 0 && !docsWithField.get(doc)) {
v2 = missingValue;
}

return Long.compare(bottom, v2);
}

@Override
public void copy(int slot, int doc) {
long v2 = currentReaderValues.get(doc);
// Test for v2 == 0 to save Bits.get method call for
// the common case (doc has value and value is non-zero):
if (docsWithField != null && v2 == 0 && !docsWithField.get(doc)) {
v2 = missingValue;
}

values[slot] = v2 > 0L ? 1L : 0L;
}

@Override
public void setBottom(final int bottom) {
this.bottom = values[bottom];
}

@Override
public void setTopValue(Long value) {
topValue = value;
}

@Override
public Long value(int slot) {
return Long.valueOf(values[slot]) ;
}

@Override
public int compareTop(int doc) {
long docValue = currentReaderValues.get(doc);
// Test for docValue == 0 to save Bits.get method call for
// the common case (doc has value and value is non-zero):
if (docsWithField != null && docValue == 0 && !docsWithField.get(doc)) {
docValue = missingValue;
}
return Long.compare(topValue, docValue);
}
}
}

 
 
其中LongComparator直接从lucene源码中copy出来，只需要做些许修改即可，最主要的修改就是copy(int slot, int doc)方法，在复制比较值得过程中，将所有存在库存的值都视为1，否则视为0，这样排序的结果就是我们所期待的。
 

我们用到的测试用例：
 

Directory directory1 = FSDirectory.open(Paths.get(
"/Users/xxx/develop/tools/solr-5.5.0/server/solr/product/data/index"));
DirectoryReader directoryReader1 = DirectoryReader.open(directory1);
IndexSearcher searcher1 = new IndexSearcher(directoryReader1);
Sort sort1 = new Sort(new SortField("psfixstock", new EmptyStockComparatorSource(), true),
new SortField("salesVolume", SortField.Type.INT, true));

TopFieldDocs topDocs1 = searcher1.search(new TermQuery(new Term("gender_text", "女士")), 10, sort1);
for (ScoreDoc scoreDoc : topDocs1.scoreDocs) {
int doc = scoreDoc.doc;
Document document = searcher1.doc(doc);
System.out.println(String.format("docId=%s, psfixstock=%s, salesVolumn=%s", doc, document.get("psfixstock"), document.get("salesVolume")));
}

 

 
在排序时，需要将其加入至Sort对象中，但执行的时候出现错误，显示docvalues的类型不正确：
 

Exception in thread "main" java.lang.IllegalStateException: unexpected docvalues type NONE for field 'psfixstock' (expected=NUMERIC). Use UninvertingReader or index with docvalues.
at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
at org.apache.lucene.index.DocValues.getNumeric(DocValues.java:227)
at org.apache.lucene.search.FieldComparator$NumericComparator.getNumericDocValues(FieldComparator.java:167)
at com.zp.solr.handler.component.EmptyStockComparatorSource$LongComparator.doSetNextReader(EmptyStockComparatorSource.java:36)
at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:812)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:744)
at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:729)
at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:671)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:577)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:627)
at com.zp.solr.handler.component.EmptyStockSortingTest.main(EmptyStockSortingTest.java:57)

 
经过一番查找，找到原因，参考文档：http://qindongliang.iteye.com/blog/2297280，我们搜索所使用到的字段没有设置对应的docType。如果在solr中，需要进行手动排序的字段，设置docValues=“true”，并进行重新索引（使用full-import方式）：
 
   

<field name="psfixstock" type="tint" indexed="true" stored="true" multiValued="false" docValues="true" />

 
 
必须要重新建立索引才可以正常运行。注意，此时Solr与Elastic Search采取的方案有所不同，Solr默认docValues=false，而ES则相反，使用Doc索引方式会对性能产生一定的影响，要谨慎使用。
 
对于lucene中，需要将添加document中增加数字类型Field：NumericDocValuesField，否则出现上面的错误，
 

document.add(new NumericDocValuesField("stock", stock));

 
 
最终的排序结果已经按照我们的需要进行了：
 

docId=2629, psfixstock=98391, salesVolumn=4685
docId=305, psfixstock=991, salesVolumn=14
docId=16762, psfixstock=3, salesVolumn=12
docId=22350, psfixstock=993, salesVolumn=10
docId=29021, psfixstock=11076, salesVolumn=10
docId=3635, psfixstock=61, salesVolumn=6
docId=4111, psfixstock=1104, salesVolumn=5
docId=10608, psfixstock=4395, salesVolumn=5
docId=4874, psfixstock=4975, salesVolumn=4
docId=4911, psfixstock=6, salesVolumn=4
docId=15071, psfixstock=998, salesVolumn=4
docId=4837, psfixstock=9, salesVolumn=3
docId=4860, psfixstock=1002, salesVolumn=3
docId=3749, psfixstock=2240, salesVolumn=2
docId=4109, psfixstock=1493, salesVolumn=2
docId=15068, psfixstock=1000, salesVolumn=2
docId=25901, psfixstock=11110, salesVolumn=2
docId=3688, psfixstock=21, salesVolumn=1
docId=4912, psfixstock=17, salesVolumn=1
docId=5035, psfixstock=2, salesVolumn=1
docId=11835, psfixstock=8, salesVolumn=1
docId=12044, psfixstock=1, salesVolumn=1
docId=13508, psfixstock=2, salesVolumn=1
docId=20019, psfixstock=1, salesVolumn=1
docId=20884, psfixstock=100000, salesVolumn=1
docId=22620, psfixstock=1, salesVolumn=1
docId=24128, psfixstock=1, salesVolumn=1
docId=0, psfixstock=2, salesVolumn=0
docId=9, psfixstock=1, salesVolumn=0
docId=11, psfixstock=4, salesVolumn=0
docId=15, psfixstock=3, salesVolumn=0
docId=20, psfixstock=4, salesVolumn=0
docId=23, psfixstock=2, salesVolumn=0
docId=24, psfixstock=5, salesVolumn=0
docId=25, psfixstock=7, salesVolumn=0
docId=35, psfixstock=2, salesVolumn=0
docId=53, psfixstock=2, salesVolumn=0