- 浏览: 739170 次
- 性别:
- 来自: 杭州
文章分类
最新评论
-
lgh1992314:
a offset: 26b offset: 24c offse ...
java jvm字节占用空间分析 -
ls0609:
语音实现在线听书http://blog.csdn.net/ls ...
Android 语音输入API使用 -
wangli61289:
http://viralpatel-net-tutorials ...
Android 语音输入API使用 -
zxjlwt:
学习了素人派http://surenpi.com
velocity宏加载顺序 -
tt5753:
谢啦........
Lucene的IndexWriter初始化时的LockObtainFailedException的解决方法
lucene的评分机制:所有hits的分数<=1.0
每个document(d)的分数:
∑tf(t in d)*idf(t)*boost(t.field in d)*lengthNorm(t.field in d)
t In q
查询的得分:
score(q,d)=coord(q,d)·queryNorm(q)·∑tf(t in d)*idf(t)*boost(t.field in d)*lengthNorm(t.field in d)
t In q
tf(t in d):文档中d出现搜索项t的频率
idf(t):搜索项t在倒排文档中出现的频率
boost(t.field in d):域的加权因子,在插入文档中设置
lengthNorm(t.field in d):域的标准化值,即在某一域中所有项的个数。通常在索引时计算该值并存储到索引文件中。
coord(q,d):协调因子(normalization value),该因子的值基于文档中包含查询项的个数
queryNorm(q):每个查询的标准化值,指每个查询项的权重的平方和
query对象的加权因子,查询时如果是多个子句,则可以通过加权某一个查询子句来加权某一个query对象。
DefaultSimilarity.java默认处理计分规则/** Expert: Default scoring implementation. */
public class DefaultSimilarity extends Similarity { /** Implemented as * <code>state.getBoost()*lengthNorm(numTerms)</code>, where * <code>numTerms</code> is {@link FieldInvertState#getLength()} if {@link * #setDiscountOverlaps} is false, else it's {@link * FieldInvertState#getLength()} - {@link * FieldInvertState#getNumOverlap()}. * * @lucene.experimental */ @Override public float computeNorm(String field, FieldInvertState state) { final int numTerms; if (discountOverlaps) numTerms = state.getLength() - state.getNumOverlap(); else numTerms = state.getLength(); return state.getBoost() * ((float) (1.0 / Math.sqrt(numTerms))); } /** Implemented as <code>1/sqrt(sumOfSquaredWeights)</code>. */ @Override public float queryNorm(float sumOfSquaredWeights) { return (float)(1.0 / Math.sqrt(sumOfSquaredWeights)); } /** Implemented as <code>sqrt(freq)</code>. */ @Override public float tf(float freq) { return (float)Math.sqrt(freq); } /** Implemented as <code>1 / (distance + 1)</code>. */ @Override public float sloppyFreq(int distance) { return 1.0f / (distance + 1); } /** Implemented as <code>log(numDocs/(docFreq+1)) + 1</code>. */ @Override public float idf(int docFreq, int numDocs) { return (float)(Math.log(numDocs/(double)(docFreq+1)) + 1.0); } /** Implemented as <code>overlap / maxOverlap</code>. */ @Override public float coord(int overlap, int maxOverlap) { return overlap / (float)maxOverlap; } // Default true protected boolean discountOverlaps = true; /** Determines whether overlap tokens (Tokens with * 0 position increment) are ignored when computing * norm. By default this is true, meaning overlap * tokens do not count when computing norms. * * @lucene.experimental * * @see #computeNorm */ public void setDiscountOverlaps(boolean v) { discountOverlaps = v; } /** @see #setDiscountOverlaps */ public boolean getDiscountOverlaps() { return discountOverlaps; } }
IndexSearcher.java的explain方法返回的Explanation对象包含了所有评分因子中各个因子的详细信息。
测试程序和数据参考http://zhwj184.iteye.com/admin/blogs/1522709
import java.io.File; import java.io.IOException; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.document.Document; import org.apache.lucene.index.IndexReader; import org.apache.lucene.queryParser.ParseException; import org.apache.lucene.queryParser.QueryParser; import org.apache.lucene.search.Explanation; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.Query; import org.apache.lucene.search.ScoreDoc; import org.apache.lucene.store.Directory; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.util.Version; public class DocSearch { private static IndexSearcher isearcher = null; public static void search(String key) throws IOException, ParseException{ Directory directory = FSDirectory.open(new File("E:\\output\\lucence\\index")); // Now search the index: IndexReader ireader = IndexReader.open(directory); // read-only=true isearcher = new IndexSearcher(ireader); // Parse a simple query that searches for "text": Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_CURRENT); QueryParser parser = new QueryParser(Version.LUCENE_CURRENT,"context", analyzer); Query query = parser.parse(key); ScoreDoc[] hits = isearcher.search(query, null, 1000).scoreDocs; // Iterate through the results: for (int i = 0; i < hits.length; i++) { Document hitDoc = isearcher.doc(hits[i].doc); System.out.println(hitDoc.getValues("id")[0] + "\t" + hitDoc.getValues("context")[0] + "\t" + hits[i].score); Explanation explan = isearcher.explain(query, hits[i].doc); System.out.println(explan); } } public static void main(String[] args) throws IOException, ParseException { search("旧水泥袋"); isearcher.close(); } }
输出结果:
只截取第一篇文档的评分信息
4801857 采购旧编织袋、旧水泥袋 4.0172114 4.0172114 = (MATCH) sum of: 1.4140004 = (MATCH) weight(context:旧 in 1682), product of: 0.54585564 = queryWeight(context:旧), product of: 5.861472 = idf(docFreq=13, maxDocs=1809) 0.09312603 = queryNorm 2.5904293 = (MATCH) fieldWeight(context:旧 in 1682), product of: 1.4142135 = tf(termFreq(context:旧)=2) 5.861472 = idf(docFreq=13, maxDocs=1809) 0.3125 = fieldNorm(field=context, doc=1682) 0.60229266 = (MATCH) weight(context:水 in 1682), product of: 0.42365694 = queryWeight(context:水), product of: 4.549286 = idf(docFreq=51, maxDocs=1809) 0.09312603 = queryNorm 1.4216518 = (MATCH) fieldWeight(context:水 in 1682), product of: 1.0 = tf(termFreq(context:水)=1) 4.549286 = idf(docFreq=51, maxDocs=1809) 0.3125 = fieldNorm(field=context, doc=1682) 1.1562659 = (MATCH) weight(context:泥 in 1682), product of: 0.58700174 = queryWeight(context:泥), product of: 6.3033047 = idf(docFreq=8, maxDocs=1809) 0.09312603 = queryNorm 1.9697827 = (MATCH) fieldWeight(context:泥 in 1682), product of: 1.0 = tf(termFreq(context:泥)=1) 6.3033047 = idf(docFreq=8, maxDocs=1809) 0.3125 = fieldNorm(field=context, doc=1682) 0.84465253 = (MATCH) weight(context:袋 in 1682), product of: 0.42188305 = queryWeight(context:袋), product of: 4.5302377 = idf(docFreq=52, maxDocs=1809) 0.09312603 = queryNorm 2.0021012 = (MATCH) fieldWeight(context:袋 in 1682), product of: 1.4142135 = tf(termFreq(context:袋)=2) 4.5302377 = idf(docFreq=52, maxDocs=1809) 0.3125 = fieldNorm(field=context, doc=1682)
发表评论
-
对字符串进行验证之前先进行规范化
2013-09-17 23:18 13879对字符串进行验证之前先进行规范化 应用系统中经常对字 ... -
使用telnet连接到基于spring的应用上执行容器中的bean的任意方法
2013-08-08 09:17 1425使用telnet连接到基于spring的应用上执行容器中 ... -
jdk7和8的一些新特性介绍
2013-07-06 16:07 10064更多ppt内容请查看:htt ... -
Lucene的IndexWriter初始化时的LockObtainFailedException的解决方法
2013-06-28 21:35 11730原文链接: http://www.javaarch.net ... -
java对于接口和抽象类的代理实现,不需要有具体实现类
2013-06-12 09:50 2904原文链接:http://www.javaarch.net/j ... -
Excel2007格式分析和XML解析
2013-06-07 09:56 10608在物料清单采购中,用到excel上传文件解析功能,不 ... -
Java EE 7中对WebSocket 1.0的支持
2013-06-05 09:27 3795原文链接:http://www.javaarch.n ... -
java QRCode生成示例
2013-06-05 09:26 1462原文链接:http://www.javaarch.n ... -
Spring Security Logout
2013-06-03 00:05 2319原文地址:http://www.javaarch.net/ ... -
Spring Security Basic Authentication
2013-06-03 00:04 1693原文地址:http://www.javaarch.net/ ... -
Spring Security Form Login
2013-06-02 16:16 2109原文地址:http://www.javaarch.net/j ... -
spring3 的restful API RequestMapping介绍
2013-06-02 14:53 1113原文链接:http://www.javaarch.net/j ... -
Java Web使用swfobject调用flex图表
2013-05-28 19:05 1076Java Web使用swfobject调用 ... -
spring使用PropertyPlaceholderConfigurer扩展来满足不同环境的参数配置
2013-05-21 15:57 3285spring使用PropertyPlaceholderCon ... -
java国际化
2013-05-20 20:57 4435java国际化 本文来自:http://www.j ... -
RSS feeds with Java
2013-05-20 20:52 1182RSS feeds with Java 原文来自:htt ... -
使用ibatis将数据库从oracle迁移到mysql的几个修改点
2013-04-29 10:40 1628我们项目在公司的大战略下需要从oracle ... -
线上机器jvm dump分析脚本
2013-04-19 10:48 2856#!/bin/sh DUMP_PIDS=`p ... -
spring3学习入门示例工程
2013-04-18 09:28 10841. github地址 https://github ... -
spring map使用annotation泛型注入问题分析
2013-04-15 13:30 8500今天在整一个spring的ioc学习demo,碰到 ...
相关推荐
lucene-3.6.0.zip
开源项目lucene-3.6.0 官网上下下来的源码
lucene-core-3.6.0-sources 绝对可用
lucene-3.6.0 api 手册, 最新的 , lucene 是个好东东, 一直在用, 之前还在使用3.1的,发现已经到3.6了, 落后啊
lucene文档,lucene文档,lucene文档,lucene文档,lucene文档,lucene相关文档
包含翻译后的API文档:lucene-core-7.2.1-javadoc-API文档-中文(简体)版.zip; Maven坐标:org.apache.lucene:lucene-core:7.2.1; 标签:apache、lucene、core、中文文档、jar包、java; 使用方法:解压翻译后的API...
lucene-core-3.6.0.jar,很好,很实用的一个包
IKAnalyzer所有的Jar包以及lucene3.6.0和lucene5.0.0相关的jar包,IKAnalyzer2012兼容lucene3.6.0,IKAnalyzer兼容lucene有限
weblucene安装文档 。。。。。。。。。。。。。。。
包含翻译后的API文档:lucene-core-7.7.0-javadoc-API文档-中文(简体)版.zip; Maven坐标:org.apache.lucene:lucene-core:7.7.0; 标签:apache、lucene、core、中文文档、jar包、java; 使用方法:解压翻译后的API...
Lucene学习文档Lucene学习文档Lucene学习文档Lucene学习文档
lucene整理文档,lucene详细描述,安装使用过程。
这是lucene的使用案例,实现了对word文档中的关键字检索,并将检索出的内容高亮打印出来
Lucene是apache软件基金会4 jakarta项目组的一个子项目,是一个开放源代码的全文检索引擎工具包,但它不是一个完整的全文检索引擎
Lucene是apache软件基金会4 jakarta项目组的一个子项目,是一个开放源代码的全文检索引擎工具包,但它不是一个完整的全文检索引擎,而是一个全文检索引擎的架构,提供了完整的查询引擎和索引引擎,部分文本分析引擎...
深入 Lucene 索引机制深入 Lucene 索引机制深入 Lucene 索引机制深入 Lucene 索引机制深入 Lucene 索引机制深入 Lucene 索引机制
这里面是lucene的相关学习资料,特别适合新手学习。
lucene索引工具包官方开发文档,适用版本4.6.0
lucene帮助文档解压之后是.chm格式的。
lucene技术文档.doc