phpbotscout index refresh¶

Re-indexes only the sources that have changed, leaving the rest alone.

Usage¶

$ phpbotscout index refresh
52 sources: 1 indexed, 51 unchanged, 0 deferred, 0 failed
80 documents, 768 chunks

What "changed" means¶

A source is re-indexed when the commit at the head of its default branch differs from the one recorded against it. The comparison is against the recorded commit, not a timestamp. A repository force-pushed backwards has changed just as much as one pushed forwards.

Everything else is counted as unchanged and reported, rather than silently omitted. A source that has never been indexed is indexed here too, so refresh picks up a new repository in the group without a full build.

The commit is read from the source's kept clone after a fast-forward, which is one round trip for a source that has not moved. A source is not fetched at all when the forge's last recorded activity on it is more than an hour older than the start of its last successful fetch. The hour is there because GitLab moves that timestamp at most once an hour, so a push within the hour can leave it unchanged. A source with no recorded activity is always fetched.

Up to two sources are fetched at once.

Failure is per-source¶

A source that cannot be cloned or read does not end the run. It is recorded against that source with its reason and the run continues, so one unreachable repository does not cost you all the others:

$ phpbotscout index refresh
52 sources: 2 indexed, 49 unchanged, 1 failed
120 documents, 900 chunks

Failed:
  phpboyscout/go/config
    clone refused

Abandoning the cycle on the first failure would leave the whole corpus as stale as its unluckiest repository. The reason is also kept against the source, so status answers "why is this stale" and not merely "this is stale".

A rate limit stops the run¶

The first time the forge refuses a fetch with a rate limit, the run stops. The fetch already in flight is cancelled and nothing further starts. Asking a forge that has said to wait is how a throttle becomes a lockout. Three fetches failing in a row stop the run too, for a limit that does not announce itself as one.

Sources the run did not reach are counted as deferred. They keep the index they had, status marks them, and the next run tries them again:

$ phpbotscout index refresh
52 sources: 3 indexed, 9 unchanged, 40 deferred, 0 failed
240 documents, 1800 chunks

Stopped early: rate limited fetching phpboyscout/go/forge. Deferred sources keep their previous index.

A stopped run does not delete the clones of sources that no longer qualify. The next run that completes does.

Relationship to `serve`¶

phpbotscout serve runs this same code on a timer. There is one implementation, so the scheduled and manual routes cannot drift into disagreeing about what is indexed or where the index lives.

Running index refresh by hand while the daemon is running is safe, but the two will contend for the database.

Discovery runs every time¶

refresh re-enumerates the namespace on every run rather than reusing the last listing. A repository made private since the last build must stop being indexed immediately, and that is only knowable by asking. See index sources for what discovery costs.

Flags¶

refresh defines no flags of its own. Global flags (--config, --debug, --output, --ci, --accessible) are inherited from the root command.

Name	Description	Default
`-h`, `--help`	Help for `refresh`.	`false`