add simple local llm (#202)A very simple local llm. Not as good as OpenAI but works as a drop-in replacement for on premise deployments.
---------
Co-authored-by: Yuhong Sun <[email protected]>
Improve indexing status display (#278)Adds:
- actual error message in UI for indexing failure
- if a connector is disabled, stops indexing immediately (after the current batch of documents) to allow for deletion
- adds num docs indexed for the current run + a speed
Add Github Action to run mypy / reorder-python-imports / black on all PRs (#251)Also fixes import ordering (previously, local imports weren't grouped together as they should have been)
Add support for openid connect (#206)This allow using Danswer in typical (non-google) enterprise environments.
* Access Tokens can be very large. A token without claims is already 1100 bytes for me (larger than allowed in danswer by default). With roles I got a 12kB token. For that reason I changed the field to TEXT in the database.
* Danswer used to swallow most errors when OIDC would fail. Nodejs forwards a request to the backe...
prevent crash when semantic_identifier is None (#201)This is a workaround around intermittent issues where sementic_identifier becomes None for some reason. It usually recovers when documents are rescraped.
Obviously, we do not yet understand the issue and are interested in a better solution.
Fix a few bugs with Google Drive polling (#250)- Adds some offset to the `start` for the Google Drive connector to give time for `modifiedTime` to propagate so we don't miss updates
- Moves fetching folders into a separate call since folder `modifiedTime` doesn't get updated when a file in the folder is updated
- Uses `connector_credential_pair.last_successful_index_time` instead of `updated_at` to determine the `start` for poll connectors
