The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

texture (13) [Avatar] Offline
The setEnablePositionIncrements method is not clear whether we should set to true or false.

From the source code of

public final class StopFilter extends TokenFilter {
private final CharArraySet stopWords;
private boolean enablePositionIncrements = false;

One can see that enablePositionIncrements is set, by default, to false, which means don't preserve the "holes".

So, are you suggesting to set explicitly to false, even thou the default is already set to false? Or am I missing any bit?
mike.mccandless (221) [Avatar] Offline
Re: setPositionIncrements, p.172 (soft pg num), sec. 4.6.1
"true" means the position will be incremented to reflect the holes left behind when stopwords are removed.

The defaulting is complicated...

In 3.0.1, StopFilter's ctor requires that you pass in enablePositionIncrements, so there's no default at the class level. However StopFilter exposes a static getEnablePositionIncrementsVersionDefault, which returns true when version is >= 2.9. Analyzers that create a StopFilter uses this to get the version-dependent default.
texture (13) [Avatar] Offline
Re: setPositionIncrements, p.172 (soft pg num), sec. 4.6.1
This explains why the AnalyzerUtils.displayTokensWithPositions works with position incrementation without the need to set the variable to true.

It is all much clear now. Thank you!