Rebase postgres branch on master #2398

…ing it to mpt imap test - The PostgresMailboxSessionMapperFactoryTODO was created independent of PostgresMailboxSessionMapperFactory for development. We need to remove MapperFactory, and rename MapperFactoryTODO -> MapperFactory when all dependencies already.

…dSafetyTest We do not control directly r2dbc-postgresql Connection but library owner, thus we can do nothing upon tests failure. But we can handle library failure at James layer using PostgresExecutor wrapper.

PreparedStatement id is unique per PG connection. We share a PG connection across multi threads leads to PreparedStatement id conflicts. We can retry upon PreparedStatement id conflicts.

…ct exception io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: [42P05] prepared statement "S_0" already exists Should not retry upon other fatal exception e.g. database failure, invalid authorization...

Runs 41 tests more onto the PostgresMailboxManager

We generate a unique UUID so this is supported. Note that turns 5 tests on in the PostgresMailboxManager test suite.

- Use single query to database for fetch total + total unseen The query looks like: select count(*) as "total_count", count(*) filter (where is_seen = $1) as "unseen_count" from message_mailbox where mailbox_id = cast($2 as uuid)

- It is a why make lack of test case when run the test (jpaGuiceServerShouldUpdateQuota)

@disabled

…ase sql validation - It was created for JPA, with Postgresql we don't need it. (instead of mark @disabled for it)

…tentLoader

Fixed by JAMES-2586 Small codestyle refactorings.

…lict Otherwise, we could retry on other fatal errors like: io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: column "domain" of relation "mailbox" already exists.

…olicy before alter the table

…iteTable

This prevents data loss.

Co-authored-by: Quan Tran <hqtran@linagora.com>

…JPA code

…lRepository

…ostgres table Would be useful in case jOOQ DSL does not provide support for some queries e.g. create an EXCLUDE constraint.

Merge a few PRs parallel led to this.

Again, an issue because of merging PRs in parallel.

While not needed now this extra field would significantly ease a future JMAP Sieve implementation and avoid a migration.

…1858)

…he#1867)

…s app

…se with only Postgresql after adding search module chooser

…e#1875)

…rties

…bitmq-guice

…eplace to invoke DAO directly (apache#1880)

…verShouldUpdateQuota Co-authored-by: Tung Van TRAN <vttran@linagora.com>

- The test is not valid because the upload parallelism with big blobs takes time and the test does not waiting for the end of the upload

…oreModulesChooser

…bling Row level security - Remove old test cases from PostgresMailboxManagerTest - Create a contract test for the listener with 2 implementations: withoutRLS and withRLS

…n enabling RLS

- The priority will determine the sort order before start initialization

- Use the InitializationOperation with higher priority to ensure it will run first (before DomainList init)

…s DeletedMessageVault

…eListener missing attachment metadata though -> need to do it in linagora#5011

Can not rely on `DeletedMessageVaultIntegrationTest` for now, as it requires `JmapGuiceProbe`.

…ovider

The module where we can implement Postgres storage layer for JMAP.

…lbox deletion

…` module

…modules - Making it easier to install the binding when a user chooses this option.

Matching logic with tests package location and cassandra mailbox module

…Mapper + tests

… posgres mailbox tests

…box has a lot of messages

… list

…Repository interface - It will more abstract for another implement (here is Postgres)

- Fix the startup docker-compose not work

In the Postgres implementation, we accept to just store the input ZonedDateTime under the UTC time zone. Therefore, we need to be more flexible comparing ZonedDateTime between different time zones. E.g.: 2014-04-02T19:01Z[UTC] isEqual 2014-04-03T02:01+07:00[Asia/Vientiane]

…ouldNotPersistWhenExpiryDateIsPresent The scenario was not really as same as the test name.

…ck the stats of the SQL statement execution

…red Offset time

…e constraint for deviceClientId (apache#2094) Avoid 2 round trips (checking duplicate deviceClientId + INSERT subscription) when saving a subscription.

…lection size (apache#2103)

…etTest

…est module

It defaults to 1.

)

These are the tests took a lot of time to fully repeat. cf: https://ge.apache.org/s/dqudut5akzxr6/tests/goal/org.apache.james:postgres-jmap-rfc-8621-integration-tests:surefire:test@default-test/details/org.apache.james.jmap.rfc8621.postgres.PostgresUploadTest?top-execution=1 cf: https://ge.apache.org/s/dqudut5akzxr6/tests/goal/org.apache.james:postgres-jmap-rfc-8621-integration-tests:surefire:test@default-test/details/org.apache.james.jmap.rfc8621.postgres.PostgresMailboxSetMethodTest?top-execution=1 Likely overkill to repeat that much, we can reduce the repeat count to save some test runtime...

- get the list replaced to get each by each

…odTest

…return Boolean value when applied

- Method delete of Upload Repository should return Boolean value when applied - optimize resetSpace method - single call replace to twice call (get & update) - The auto cleanup upload when exceed do not ensure the concurrent -> we need sleep time after concurrent upload.

…x to avoid noise log (Mailbox and User table)

- Do not print log "Table {} created", "Index {} created" when it already exists and James does nothing

…tgres Message in order to remove duplicated code: - Introduce PostgresMessageRetriever (= AttachmentLoader + Retrieve byte message content)

…e places Was declared in the backend-common postgres module, and it is enough.

…hase cf: apache@c6f2462

…cassandra to quota-mailing module

…d classes from quota-mailing module

…ng Tran 10 minutes ago

…ent Value Revert commit: 721f9c0 Reason: when provision messages mailbox, we have a lot of error related to: `quota_current_value_primary_key` The old way helps us avoid that

This reverts commit 3bb549a. Reason: the provision not works. The `getCurrentQuota` alway return the "ZERO" value (not empty)

…ZonedDateTime(s) with different timezones

TODO handle [Guice/CanNotProxyClass]: Tried proxying JsonTaskSerializer to support a circular dependency, but it is not an interface. in another ticket.

Use RabbitMQ mail queue instead.

…postgres-app

apache#2179)

jooq reactive bug: SELECT many records then query something else would hang This is a temporary fix in the meantime waiting for the jooq fix. Bear in mind that `.collectList` lot of elements could impact performance.

…r (remove Jmap draft)

…ctionFactory

…ng connection pool

…nFactory

…g set-domain command in case of empty domain

…ipeline

…gresDeletedMessageVaultDeletionCallback (apache#2280) To fixing exception: java.lang.ClassCastException: class org.apache.james.vault.metadata.DeletedMessageVaultDeletionCallback cannot be cast to class org.apache.james.mailbox.postgres.DeleteMessageListener$DeletionCallback (org.apache.james.vault.metadata.DeletedMessageVaultDeletionCallback and org.apache.james.mailbox.postgres.DeleteMessageListener$DeletionCallback are in unnamed module of loader 'app')

* [BUILD] Increase jOOQ reactive timeout for testing After Apache James CI issue with infra recently, the tests are somehow unstable. PostgresBlobStoreDAOTest.concurrentSaveInputStreamShouldReturnConsistentValues: ``` 18:39:45.087 [ERROR] o.a.j.b.p.u.PostgresExecutor - Time out executing Postgres query. May need to check either jOOQ reactive issue or Postgres DB performance. java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 10000ms in 'flatMap' (and no fallback has been configured) ``` Note that the failure rate locally is really low though... * JAMES-2586 - Override duration timeout for concurrent test in PostgresBlobStoreDAOTest Co-authored-by: Tung Van TRAN <vttran@linagora.com> --------- Co-authored-by: Tung Van TRAN <vttran@linagora.com>

- Using PoolBackedPostgresConnectionFactory for all factory. Make it similar with prod code

…nection api

…RELEASE

… and email_change

…MailboxMapper -create new table MailboxMember (username, mailbox_id) -create dao for new table -create RLSSupportPostgresMailboxMapper to use MailboxMember -update bindings to use RLSSupportPostgresMailboxMapper in case rls is enabled

…gresMailboxMapper - create index for mailbox_acl column in case rls is disabled - update findNonPersonalMailboxes method in PostgresMailboxMapper and PostgresMailboxDAO

…thod - Update AdditionalAlterQuery in PostgresTable - check if upsertACL actually successful - replace batch method - remove duplicated code in PostgresMailboxMapper and RLSSupportPostgresMailboxMapper

…ipeline - Update: disabled for cassandra weakWrite

…xtension

test failed, and CassandraJamesServerMain was mark as deprecated, and will be removed in the future.

…abbitmq-object-storage

…ds documentation page

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebase postgres branch on master #2398

Rebase postgres branch on master #2398

Commits on Aug 30, 2024

Rebase postgres branch on master #2398

Are you sure you want to change the base?

Rebase postgres branch on master #2398

Commits on Aug 30, 2024