This project has moved and is read-only. For the latest updates, please go here.
1

Closed

Scan with a filter terminates if result page has 0 matches, even when the page is not last page of scan results

description

Scan with a filter terminates if result page has 0 matches, even when the page is not the last page of scan results.

The search reader must continue reading through subsequent pages while the response from Dynamo includes last evaluated key.
Closed Jan 17, 2015 at 10:34 PM by scale_tone

comments

scale_tone wrote Jan 12, 2015 at 9:04 PM

Thank you for reporting this, but maybe just a small piece of code, that demonstates the issue?
How is that generally possible: get a page with 0 records, while this page is not the last page in the results?

AgentKay wrote Jan 12, 2015 at 10:39 PM

I don't think I can give a concise code example, but let me try to explain it.

When you do a scan with a filter condition, the filter is evaluated on the server side and the page is returned after evaluating certain number of records. If neither of the records matches specified filter condition, the resulting page will contain 0 records, although the scan is not finished (the last evaluated key is specified).

I already fixed this in my local repo, the code for the fixed method is below:
See Readers/SearchReader.cs
private bool SearchResultModeMoveNext()
            {
                if
                (
                    (this._currentBatch == null)
                    ||
                    (this._currentBatchIndex >= this._currentBatch.Count)
                )
                {
                    if (this._search.IsDone)
                    {
                        return false;
                    }

                    // read though results until we reach the end or get at least one document.
                    do
                    {
                        this._currentBatch = this._search.GetNextSet();

                        if (this._search.IsDone && this._currentBatch.Count == 0)
                        {
                            return false;
                        }
                    }
                    while (!this._search.IsDone && this._currentBatch.Count == 0);

                    this._currentBatchIndex = 0;
                }

                var currentDoc = this._currentBatch[this._currentBatchIndex++];

                this.NewEntityDocumentEnumerated.FireSafely(currentDoc, this._entityType);

                if (this._projectionFunc == null)
                {
                    this.Current = (TEntity)this.LoadOrCreateEntityDocument.FireSafely(currentDoc);
                }
                else
                {
                    this.Current = this._projectionFunc(currentDoc);
                }

                return true;
            }
As you can see, I fixed termination condition to use IsDone flag instead of testing the number of results on the page.

scale_tone wrote Jan 13, 2015 at 1:55 PM

Yeah, now I see, it's a bug indeed...

wrote Jan 17, 2015 at 10:34 PM

Resolved with changeset 38539: Fix for issue 1527