Rethink softCommit=true|false param on commits?
2014-01-27 10:22
393 查看
评:原文链接见:https://issues.apache.org/jira/browse/SOLR-3539
这篇文章对理解openSearcher和softcommit两个参数的含义非常有帮助,
并预计会在Solr5.0中实现新的参数组合flush/openSearcher以降低混淆性。
[SOLR-3539]
I think current NRT options when doing a commit, particularly "openSearcher="true|false" and "softCommit=true|false", is confusing, we should rethink them before they get into user API in 4.0.
SOLR-4370 Ability to control the commit behaviour of commitWithin
All comments:
Hoss Man
added a comment - 12/Jun/12 21:02
This is something that started to concern me while trying to update the tutorial. I'm having a hard time articulating my concerns to myself, so this will largely be stream of consciousness...
Both of these params seen defined more in terms of what they don't do then what they actually do – softCommit in particular – and while they aren't too terrible to explain indivdually, it's very hard to clearly articulate how they interplay
with eachother.
openSearcher
true - opens a new searcher against this commit point
false - does not open a new searcher against this commit point
softCommit
true - a new searcher is opened against the commit point, but no data is flushed to disk.
false - the commit point is flushed to disk.
Certain combinations of these params seem redundent (openSearcher=true&softCommit=true) while others not only make no sense, but are directly contradictory (openSearcher=false&softCommit=true)...
From a vocabulary standpoint, they also seem confusing to understand. Consider a new user, starting with the 4x example which contains the following...
Documents this user adds will automaticly get flushed to disk, but won't be visible in search results until the user takes some explicit action. The user, upon reading some docs or asking on the list will become aware that he needs to open a new searcher,
and will be guided to "do a commit" (or maybe a commit explicitly adding openSearcher=true). But this is actually overkill for what the user needs, because it will also flush any pending docs to disk. All the user really needs to "open a new searcher" is to
do an explicit commit with softCommit=true.
I would like to suggest that we throw out the the "softCommit" param and replace it with a "flush" (or "flushToDisk" or "persist") param, which is solely concerned with the persistence of the commit, and completely disjoint from "searcher" opening which
would be controled entirely with the "openSearcher" param.
openSearcher
true - opens a new searcher against this commit point
false - does not open a new searcher against this commit point
flush
true - flushes this commit point to stable storage
false - does not flush this commit point to stable storage
Making the interaction much easier to understand...
I've mainly been thinking about this from a user perspective the last few days, so I haven'thad a chance to figure out how much this would impact the internals related to softCommit right now. I supsect there are a lot of places that would need to be tweaked,
but hopefully most of them would just involve flipping logic (softCommit=true -> flush=false). The biggest challenges i can think of are:
how to deal with the autocommit options in solrconfig.xml. in 3x we supported a single <autoCommit/> block. On the 4x branch we support one <autoCommit/> lock and one <autoSoftCommit/> block – should we continue to do that? would <autoSoftCommit/> just
implicitly specify flush=false? or should we try to generalize to support N <autoCommit/> blocks where <openSearcher/> and <flush/> are config options for all of them?
event eventlistener – it looks like the SolrEventListener API had a postSoftCommit() method added to it, but it doesn't seem to be configurable in any way – i think this is just for tests, but if it's intentionally being expost we would need to revamp it
... off the cuff i would suggest removing postSoftCommit() changing the postCommit() method to take in some new structure specifying the options on the commit.
Thoughts?
Hoss Man
added a comment - 11/Jul/12 23:25
bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
Yonik Seeley added a comment - 27/Jul/12 11:46
Anyone else have thoughts around this?
One performance concern of mine revolves around "commit" - the vast majority of people used it for visibility of documents, not for persistence at a specific time.
I'm warming to the idea of a "flush" param instead of softCommit, and it seems like perhaps it should default to "false" for 4.0
Robert Muir
added a comment - 07/Aug/12 04:43
rmuir20120906-bulk-40-change
Mark Miller added a comment - 27/Aug/12 19:44
I agree we could clean this up.
I worry about flush since it used to mean something else.
Hoss Man
added a comment - 27/Aug/12 19:53
I worry about flush since it used to mean something else.
"persist" ?
Robert Muir
added a comment - 15/Sep/12 13:49
Unassigned issues -> 4.1
Steve
Rowe added a comment - 23/Jul/13 19:47
Bulk move 4.4 issues to 4.5 and 5.0
这篇文章对理解openSearcher和softcommit两个参数的含义非常有帮助,
并预计会在Solr5.0中实现新的参数组合flush/openSearcher以降低混淆性。
[SOLR-3539]
I think current NRT options when doing a commit, particularly "openSearcher="true|false" and "softCommit=true|false", is confusing, we should rethink them before they get into user API in 4.0.
Issue Links
related:SOLR-4370 Ability to control the commit behaviour of commitWithin
All comments:
Hoss Man
added a comment - 12/Jun/12 21:02
This is something that started to concern me while trying to update the tutorial. I'm having a hard time articulating my concerns to myself, so this will largely be stream of consciousness...
Both of these params seen defined more in terms of what they don't do then what they actually do – softCommit in particular – and while they aren't too terrible to explain indivdually, it's very hard to clearly articulate how they interplay
with eachother.
openSearcher
true - opens a new searcher against this commit point
false - does not open a new searcher against this commit point
softCommit
true - a new searcher is opened against the commit point, but no data is flushed to disk.
false - the commit point is flushed to disk.
Certain combinations of these params seem redundent (openSearcher=true&softCommit=true) while others not only make no sense, but are directly contradictory (openSearcher=false&softCommit=true)...
softCommit=true | softCommit=false | |
openSearcher=true | openSearcher is redundent | OK |
openSearcher=false | contradictory (openSearcher is currently ignored) | OK |
<autoCommit> <maxTime>15000</maxTime> <openSearcher>false</openSearcher> </autoCommit>
Documents this user adds will automaticly get flushed to disk, but won't be visible in search results until the user takes some explicit action. The user, upon reading some docs or asking on the list will become aware that he needs to open a new searcher,
and will be guided to "do a commit" (or maybe a commit explicitly adding openSearcher=true). But this is actually overkill for what the user needs, because it will also flush any pending docs to disk. All the user really needs to "open a new searcher" is to
do an explicit commit with softCommit=true.
I would like to suggest that we throw out the the "softCommit" param and replace it with a "flush" (or "flushToDisk" or "persist") param, which is solely concerned with the persistence of the commit, and completely disjoint from "searcher" opening which
would be controled entirely with the "openSearcher" param.
openSearcher
true - opens a new searcher against this commit point
false - does not open a new searcher against this commit point
flush
true - flushes this commit point to stable storage
false - does not flush this commit point to stable storage
Making the interaction much easier to understand...
flush=true | flush=false | |
openSearcher=true | OK | OK |
openSearcher=false | OK | No-Op |
but hopefully most of them would just involve flipping logic (softCommit=true -> flush=false). The biggest challenges i can think of are:
how to deal with the autocommit options in solrconfig.xml. in 3x we supported a single <autoCommit/> block. On the 4x branch we support one <autoCommit/> lock and one <autoSoftCommit/> block – should we continue to do that? would <autoSoftCommit/> just
implicitly specify flush=false? or should we try to generalize to support N <autoCommit/> blocks where <openSearcher/> and <flush/> are config options for all of them?
event eventlistener – it looks like the SolrEventListener API had a postSoftCommit() method added to it, but it doesn't seem to be configurable in any way – i think this is just for tests, but if it's intentionally being expost we would need to revamp it
... off the cuff i would suggest removing postSoftCommit() changing the postCommit() method to take in some new structure specifying the options on the commit.
Thoughts?
Hoss Man
added a comment - 11/Jul/12 23:25
bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
Yonik Seeley added a comment - 27/Jul/12 11:46
Anyone else have thoughts around this?
One performance concern of mine revolves around "commit" - the vast majority of people used it for visibility of documents, not for persistence at a specific time.
I'm warming to the idea of a "flush" param instead of softCommit, and it seems like perhaps it should default to "false" for 4.0
Robert Muir
added a comment - 07/Aug/12 04:43
rmuir20120906-bulk-40-change
Mark Miller added a comment - 27/Aug/12 19:44
I agree we could clean this up.
I worry about flush since it used to mean something else.
Hoss Man
added a comment - 27/Aug/12 19:53
I worry about flush since it used to mean something else.
"persist" ?
Robert Muir
added a comment - 15/Sep/12 13:49
Unassigned issues -> 4.1
Steve
Rowe added a comment - 23/Jul/13 19:47
Bulk move 4.4 issues to 4.5 and 5.0
相关文章推荐
- Faceting on a field in SolrCloud with docValues=true and indexed=false fails
- HINT避免MERGE JOIN/*+ OPT_PARAM(\'_optimizer_mjc_enabled\',\'false/true\')
- setOnTouchEvent 设置返回值为true 和 false的区别
- 【HTML/CSS】 JS中form表单中的onsubmit监听返回 true或者false用法
- [android] setOnTouchEvent 设置返回值为true 和 false的区别
- setOnTouchEvent 设置返回值为true 和 false的区别
- Android------三种监听OnTouchListener、OnLongClickListener同时实现即其中返回值true或者false的含义
- [D3] 11. Basic D3 chart interactivity on(), select(this), classed(class, trueorfalse)
- 监听OnTouchListener、OnLongClickListener中返回值true或者false的含义
- conn.setAutoCommit(true)和(false)的区别
- conn.setAutoCommit(true)和(false)的区别
- conn.setAutoCommit(true)和(false)的区别
- conn.setAutoCommit(true)和(false)的区别
- 浅析Android的setOnTouchListener的返回值是false和true的区别
- JS return false 与 return true
- Recall(召回率);Precision(准确率);F1-Meature(综合评价指标);true positives;false positives;false negatives..
- jquery checkbox反复调用attr('checked', true/false)只有第一次生效
- request.getSession(true)和request.getSession(false)的区别
- Ajax请求中的async:false/true的作用
- MFC-UpdateData(false)和UpdateData(true)的区别