Commit Graph

60 Commits

Author SHA1 Message Date
Daniel Dietzler
34ce68095d chore: upgrade to kysely 0.28.11 (#26744) 2026-03-11 16:17:31 +01:00
Kang
02b29046b3 feat: ocr (#18836)
* feat: add OCR functionality and related configurations

* chore: update labeler configuration for machine learning files

* feat(i18n): enhance OCR model descriptions and add orientation classification and unwarping features

* chore: update Dockerfile to include ccache for improved build performance

* feat(ocr): enhance OCR model configuration with orientation classification and unwarping options, update PaddleOCR integration, and improve response structure

* refactor(ocr): remove OCR_CLEANUP job from enum and type definitions

* refactor(ocr): remove obsolete OCR entity and migration files, and update asset job status and schema to accommodate new OCR table structure

* refactor(ocr): update OCR schema and response structure to use individual coordinates instead of bounding box, and adjust related service and repository files

* feat: enhance OCR configuration and functionality

- Updated OCR settings to include minimum detection box score, minimum detection score, and minimum recognition score.
- Refactored PaddleOCRecognizer to utilize new scoring parameters.
- Introduced new database tables for asset OCR data and search functionality.
- Modified related services and repositories to support the new OCR features.
- Updated translations for improved clarity in settings UI.

* sql changes

* use rapidocr

* change dto

* update web

* update lock

* update api

* store positions as normalized floats

* match column order in db

* update admin ui settings descriptions

fix max resolution key

set min threshold to 0.1

fix bind

* apply config correctly, adjust defaults

* unnecessary model type

* unnecessary sources

* fix(ocr): switch RapidOCR lang type from LangDet to LangRec

* fix(ocr): expose lang_type (LangRec.CH) and font_path on OcrOptions for RapidOCR

* fix(ocr): make OCR text search case- and accent-insensitive using ILIKE + unaccent

* fix(ocr): add OCR search fields

* fix: Add OCR database migration and update ML prediction logic.

* trigrams are already case insensitive

* add tests

* format

* update migrations

* wrong uuid function

* linting

* maybe fix medium tests

* formatting

* fix weblate check

* openapi

* sql

* minor fixes

* maybe fix medium tests part 2

* passing medium tests

* format web

* readd sql

* format dart

* disabled in e2e

* chore: translation ordering

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-10-27 14:09:55 +00:00
Dag Stuan
78fb815cdb feat(web): add search filter for camera lens model. (#21792) 2025-10-24 14:41:34 -04:00
Arthur Normand
37a79292c0 feat: view similar photos (#21108)
* Enable filteing by example

* Drop `@GenerateSql` for `getEmbedding`?

* Improve error message

* PR Feedback

* Sort en.json

* Add SQL

* Fix lint

* Drop test that is no longer valid

* Fix i18n file sorting

* Fix TS error

* Add a `requireAccess` before pulling the embedding

* Fix decorators

* Run `make open-api`

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2025-09-04 09:22:09 -05:00
Alwin Lohrie
ae1d60e259 feat: find large files utility (#18040)
feat: large asset utility

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2025-07-28 18:48:39 -04:00
Jason Rasmussen
e73abe0762 refactor: enum casing (#19946) 2025-07-15 14:50:13 -04:00
Jason Rasmussen
c699df002a feat: rename schema (#19891) 2025-07-14 10:13:06 -04:00
Léopold Koprivnik
f778adea92 feat: adds option to search only for untagged assets (#19730)
Co-authored-by: SkwalExe <skwal@skwal.net>
2025-07-10 16:28:20 +02:00
Jason Rasmussen
e60bc3c304 refactor: database types (#19624) 2025-06-30 13:19:16 -04:00
Jason Rasmussen
5cd186d3d4 refactor: duplicate queries (#19136) 2025-06-12 14:23:02 -04:00
xCJPECKOVERx
14d785cec9 feat(server): Add album filter to search (#18985)
* - updated dtos
- added inAlbums to search builder
- only check isNotInAlbum if albumIds is blank/empty

* - consider inAlbums as OR

* - make open-api-dart

* - lint & format

* - remove inAlbums groupBy clause

* - merge main open-api

* - make open-api

* - inAlbums filter AND instead of OR
2025-06-09 11:11:43 -04:00
Jonathan Gilbert
fb4be6e231 feat(server): add /search/statistics resource (#18885) 2025-06-06 21:12:53 -04:00
Mert
fa22e865a4 fix(server): tighten asset visibility (#18699)
* tighten visibility

* update sql

* elevated access util function

* fix potential sync issue

* include in user stats

* include hidden assets in size usage

* filter visibility in search duplicates query

* stack visibility
2025-06-02 09:33:08 -05:00
Mert
0d773af6c3 feat: vectorchord (#18042)
* wip

auto-detect available extensions

auto-recovery, fix reindexing check

use original image for ml

* set probes

* update image for sql checker

update images for gha

* cascade

* fix new instance

* accurate dummy vector

* simplify dummy

* preexisiting pg docs

* handle different db name

* maybe fix sql generation

* revert refreshfaces sql change

* redundant switch

* outdated message

* update docker compose files

* Update docs/docs/administration/postgres-standalone.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* tighten range

* avoid always printing "vector reindexing complete"

* remove nesting

* use new images

* add vchord to unit tests

* debug e2e image

* mention 1.107.2 in startup error

* support new vchord versions

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2025-05-20 08:36:43 -05:00
Alex
d33ce13561 feat(server): visibility column (#17939)
* feat: private view

* pr feedback

* sql generation

* feat: visibility column

* fix: set visibility value as the same as the still part after unlinked live photos

* fix: test

* pr feedback
2025-05-06 12:12:48 -05:00
Jason Rasmussen
1fde02ee1e chore: remove unused types and code (#17999) 2025-04-30 13:41:23 -04:00
Thomas
3ce353393a chore(server): don't insert embeddings if the model has changed (#17885)
* chore(server): don't insert embeddings if the model has changed

We're moving away from the heuristic of waiting for queues to complete. The job
which inserts embeddings can simply check if the model has changed before
inserting, rather than attempting to lock the queue.

* more robust dim size update

* use check constraint

* index command cleanup

* add create statement

* update medium test, create appropriate extension

* new line

* set dimension size when running on all assets

* why does it want braces smh

* take 2

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-04-29 14:23:01 -04:00
Daniel Dietzler
dd1fcd5be5 chore: remove asset entity (#17703) 2025-04-18 21:39:56 +00:00
Daniel Dietzler
52ae06c119 refactor: remove album entity, update types (#17450) 2025-04-18 23:10:34 +02:00
Jason Rasmussen
8da5f21fcf refactor: medium tests (#17634) 2025-04-15 15:54:23 -04:00
Jason Rasmussen
b710ad36f3 feat: upgrade kysely (#17630)
* feat: upgrade kysely

* chore: pr feedback
2025-04-15 13:26:56 -04:00
Mert
25f2b9602f refactor(server): remove face, person and face search entities (#17535)
* remove face, person and face search entities

update tests and mappers

check if face relation exists

update sql

unused imports

* pr feedback

generate sql, remove unused imports
2025-04-11 14:44:45 -04:00
Jason Rasmussen
8203b6c450 refactor: stop using geodata entity type (#17444) 2025-04-08 00:15:43 -04:00
Abhinav Valecha
b621281351 feat(server): Avoid face match with people born after file creation #4743 (#16918)
* feat(server): Avoid face matching with people born after file creation date (#4743)

* lint

* add medium tests for facial recognition

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2025-04-02 10:37:26 -05:00
Jason Rasmussen
2cdbb0a37c refactor: database repository (#16593)
* refactor: database repository

* fix error reindex check

* chore: remove WIP code

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-03-06 13:33:24 -05:00
Mert
a708649504 fix(server): skip stacked assets in duplicate detection (#16380)
* skip stacked assets in duplicate detection

* update sql

* handle stacking after duplicate detection runs
2025-02-27 10:16:13 -06:00
Jason Antwi-Appah
34b88bb47a feat(web): support searching by EXIF rating (#16208)
* Add rating to search DTO

* Add search by EXIF rating in search query builder

* Generate OpenAPI spec

* Add rating filter on web

* Add rating filter to search docs

* Format / lint

* Hide rating filter if ratings are disabled

* chore: component order in form

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-02-20 16:17:06 +00:00
Jason Rasmussen
9d85272c2b refactor: repositories (#16036) 2025-02-11 14:08:13 -05:00
sudbrack
206412267a fix(server): /search/random API returns same assets every call (#15682)
* Fix for server searchRandom function not returning random results

* Fix lint
2025-01-26 14:06:18 +00:00
Mert
72fa31f9e9 fix(server): changing vector dim size (#15630) 2025-01-24 20:01:24 -05:00
Mert
ba105d9f19 fix(server): searchRandom response (#15580)
* fix searchRandom

* add e2e

* set outer limit
2025-01-24 00:41:54 -05:00
Jason Rasmussen
d3446f3092 refactor: logging repository (#15540) 2025-01-23 08:31:30 -05:00
Daniel Dietzler
332a865ce6 refactor: migrate person repository to kysely (#15242)
* refactor: migrate person repository to kysely

* `asVector` begone

* linting

* fix metadata faces

* update test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-01-21 13:12:28 -05:00
Mert
2e12c46980 refactor(server): use kysely (#12857) 2025-01-09 11:15:41 -05:00
Mert
25ca3b1124 refactor(server): use includeNull in query for search suggestions (#14626)
* use `includeNull`

* push down `includeNull` into query, inner joins

* remove filter

* update sql

* fix tests

* maybe fix e2e

* more e2e tests

* handle no exif row

* whoops

* update sql
2024-12-10 16:22:37 -05:00
Mert
34fae31fd4 fix(server): remove unnecessary guc settings for vector search (#14237)
remove unnecessary guc settings
2024-11-19 15:37:39 -06:00
Jason Rasmussen
64831e2328 refactor: remove smart info table (#13985) 2024-11-07 11:25:10 -05:00
Jason Rasmussen
e6a666f1d3 refactor(server): telemetry (#13588)
refactor: telemetry
2024-10-21 23:52:30 +00:00
Mert
063969ca05 fix(server): searching with both personIds and withPeople (#13254)
* use cte

* linting
2024-10-07 15:44:04 -04:00
Jason Rasmussen
3ac00b0ffa refactor(server): db env (#13167) 2024-10-03 17:48:40 -04:00
Mert
7adb35e59e fix(server): /search/random failing with certain options (#13040)
* fix relation handling, remove pagination

* update api, sql

* update mock
2024-09-30 00:29:35 -04:00
Jason Rasmussen
12da250028 refactor: enums (#12988) 2024-09-27 10:28:42 -04:00
jschwalbe
9f8a7e0bea feat(server): sort assets randomly from the API 'api/search/metadata' endpoint by including 'order': 'rand' in the API call. (#12741)
feat(server): search metadata random sort order

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-09-23 12:09:26 -04:00
Jason Rasmussen
4f25cec6df refactor(server): metadata repository (#12759) 2024-09-18 08:44:22 -04:00
Jason Rasmussen
b288241a5c refactor(server): enums (#11809) 2024-08-15 06:57:01 -04:00
Mert
4ed75f2ac9 refactor(server): add config events for clip (#11575)
use config events for clip, add tests

formatting
2024-08-04 21:00:36 +00:00
Mert
42f3b50422 fix(server): /places entries sometimes not ordered alphabetically (#10514) 2024-06-20 23:48:19 -04:00
Mert
79a8ab71ef fix(server): reindex after changing to a model with a different dimension size (#10496)
reindex after truncating
2024-06-19 17:25:02 -04:00
Mert
6b1b5054f8 feat(server): separate face search relation (#10371)
* wip

* various fixes

* new migration

* fix test

* add face search entity, update sql

* update e2e

* set storage to external
2024-06-16 19:25:27 +00:00
Mert
e7c8501930 fix(server): search duplicates of the same asset type (#9747)
* search by type

* make sql

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-26 22:04:23 +00:00