Commit Graph

84 Commits

Author SHA1 Message Date
Christian Grothoff
9580dd19c2
intermediate step in major SQL refactoring (not done at all) 2022-11-27 00:16:00 +01:00
Marco Boss
45916b992b
remove built sql files from vc 2022-03-31 21:49:35 +02:00
Marco Boss
75c1e10a67
add shard init for shard node 2022-03-31 21:31:07 +02:00
Marco Boss
0cf9a9984c
add sharding logic 2022-03-31 17:00:44 +02:00
Christian Grothoff
1f1d675f3d
-remove tiny field, no longer needed 2022-03-27 14:34:44 +02:00
Christian Grothoff
d0a69da895
towards removing tiny bit 2022-03-27 10:32:28 +02:00
Christian Grothoff
098d572471
change sharding strategy for refund table 2022-03-26 10:47:10 +01:00
Christian Grothoff
783e2ae424
add partitions to new p2p tables 2022-03-26 09:00:19 +01:00
Christian Grothoff
b856d56d95
rework deposits sharding, towards making aggregator faster (not necessarily done) 2022-03-24 17:33:29 +01:00
Christian Grothoff
c782dfe2aa
first draft of service API for p2p payments 2022-03-24 13:06:04 +01:00
Christian Grothoff
f091808425
more skeleton logic for P2P operations on DB 2022-03-23 12:25:45 +01:00
Christian Grothoff
41399bc224
revise P2P signatures and signing schema to address recently discovered design concern resulting in the split of the purse key into purse-contract and purse-merge keys 2022-03-23 06:54:43 +01:00
Christian Grothoff
734b68d45e
-fix drop SQL issue: drop functions properly 2022-03-22 05:33:08 +01:00
Christian Grothoff
e200e86032
implement helper functions for p2p signatures, clean up existing signature logic 2022-03-22 02:33:51 +01:00
Christian Grothoff
b560527ee3
ideas for mboss 2022-03-20 16:52:38 +01:00
Christian Grothoff
28203913e2
add p2p tables from dd13 2022-03-20 14:03:07 +01:00
Christian Grothoff
f5e5f4b843
more work on recoup_by_reserve: add new materialized index to avoid hitting all partitions for the query 2022-03-19 14:05:45 +01:00
Christian Grothoff
f22125ce0f
add missing index, move indices closer to table generation 2022-03-19 12:58:37 +01:00
Christian Grothoff
badfde8962
add deposits_by_coin table to speed-up certain queries (in theory) 2022-03-18 15:47:25 +01:00
Christian Grothoff
142a369e43
add selection by shard, ignore KYC in revocation test aggregation 2022-03-18 08:54:08 +01:00
Christian Grothoff
23af6c3c41
change table structures again to shard better by coin_pub and not use known_coin_id so much 2022-03-18 01:57:39 +01:00
Christian Grothoff
bab36a2ac8
-remove dead field, return h_commitment_age to auditor 2022-03-17 14:45:12 +01:00
Christian Grothoff
12809b28ec
-misc fixes 2022-03-17 14:16:34 +01:00
Christian Grothoff
a8d80d519d
attempt to make GRID5K_MARCO_OPT superfluous 2022-03-14 07:34:21 +01:00
Christian Grothoff
5fa977cc05
work on global fee structure 2022-03-05 15:13:07 +01:00
Christian Grothoff
7ff58c3d8f
refactor /wire to include logic to return the wad fee (for W2W payments) 2022-03-05 14:36:55 +01:00
Christian Grothoff
1643b745af
use 32 byte hash for hpayto, use that for joins on queries to better align queries with partitions 2022-03-03 23:52:08 +01:00
Marco Boss
2abe9bf6d7
include partitioning logic in dbinit 2022-03-02 10:50:51 +01:00
Marco Boss
b4f7fef224
stay with partitions for now - find shards on dev/boss/sharding 2022-03-01 15:54:47 +01:00
Marco Boss
0eed0c0de2
tested shard creation locally 2022-03-01 14:01:42 +01:00
Marco Boss
afb1b5f902
prepare sharding / partitioning init
- create exchange-0001.sql with functions to create all tables and partitions / shards
- create exchange-0002.sql for the master db
2022-03-01 12:11:58 +01:00
Christian Grothoff
ee79f31275
-work on test_sync.sh, incomplete 2022-02-28 22:41:24 +01:00
Marco Boss
0efc7fd524
cleaner sql 2022-02-28 22:32:59 +01:00
Christian Grothoff
23bc09fe3c
get test-auditor and #7181 to pass 2022-02-28 20:37:19 +01:00
Marco Boss
58eb3d95fc
towards adding shard/partitioning init functionality 2022-02-27 21:23:20 +01:00
Marco Boss
99dd36f7f6
add index on wire_targets
- better performance on local db
- many queries use wire_target_serial_id as join condition -> discussion use this as shard key?
2022-02-26 17:37:52 +01:00
Marco Boss
3d52f52164
Fix ON CONFLICT statements for distributed tables 2022-02-24 14:31:19 +01:00
Christian Grothoff
10b9023575
-on conflict example 2022-02-24 10:16:10 +01:00
Christian Grothoff
5e5004020e
-fix aggregator with deferred constraint 2022-02-24 10:06:43 +01:00
Christian Grothoff
e253a5b055
-unique constraint is crucial 2022-02-24 09:06:15 +01:00
26158fc725
[age restriction] progress 16/n - refresh/reveal/link tests
Age restriction works now with withdraw, melt/refresh/reveal and link,
including tests.

However, there is still a problem with the tests:  The melting operation
"refresh-melt-failing-age" that should fail (because of conflict), but
currently fails for other reasons.  I decided to disable that particular
test (and the next) and submit the patch I have so far.
2022-02-22 14:34:47 +01:00
Christian Grothoff
22fe5da700
-migrate to Postgress 11: use BIGINT GENERATED BY DEFAULT AS IDENTITY instead of BIGSERIAL 2022-02-17 23:08:34 +01:00
Christian Grothoff
721c2c88d5
-use age_mask 2022-02-17 15:23:56 +01:00
Christian Grothoff
a351bfc4b4
-fix CS nonce reuse check logic 2022-02-17 15:18:09 +01:00
8bdf6ab19d
[age restriction] progress 14/n - withdraw and deposit
Age restriction support for
  - withdraw is done and tested
  - deposit is done and tested

TODOs:
  - melt/refresh/reveal
  - link

------

Added functions
 - TALER_age_restriction_commit
 - TALER_age_commitment_derive
 - TALER_age_commitment_hash
 - TALER_age_restriction_commitment_free_inside
 - Hash of age commitment passed around API boundaries

Exchangedb adjustments for denominations
 - all prepared statements re: denominations now handle age_mask
 - signature parameters adjusted

Hash and signature verification of /keys adjusted
 - Hashes of (normal) denominations and age-restricted denominations are
   calculated seperately
 - The hash of the age-restricted ones will then be added to the other
   hash
 - The total hash is signed/verified

Tests for withdraw with age restriction added
 - TALER_EXCHANGE_DenomPublickey now carries age_mask
 - TALER_TESTING_cmd_withdraw_amount* takes age parameter
 - TALER_TESTING_find_pk takes boolean age_restricted
 - WithdrawState carries age_commitment and its hash
 - withdraw_run derives new age commitment, if applicable
 - Added age parameter to testing (13 as example)

Various Fixes and changes
 - Fixes of post handler for /management/extensions
 - Fixes for offline tool extensions signing
 - Slight refactoring of extensions
 - Age restriction extension simplified
   - config is now global to extension
   - added global TEH_age_restriction_enabled and TEH_age_mask in
     taler-exchange-httpd
   - helper functions and macros introduced
2022-02-16 22:01:05 +01:00
Christian Grothoff
ef938e0f7a
-correctly implement CS idempotency check on withdraw 2022-02-15 17:07:13 +01:00
Christian Grothoff
532d4ad0dc
-fixes to tests, and half-baked fixes for CS-/link (still fails) 2022-02-10 23:39:00 +01:00
0b56de6c99
[age restriction] progress 12/n
- taler-offline-tool now handles extensions
  - command "extensions" added with subcommands "show" and "sign"
  - parses extensions from taler config
  - shows and signs of extensions and their configurations
  - creates signed set of configurations for upload
  - added test for retrieval of extension config

- simplified signature verification for extensions
  - remove per-extension signatures, also from DB schema
  - adjust prepared statements accordingly
  - adjust DB event handler for extensions
  - allow NULL for config for extension in DB schema
  - handler for /management/extensions adjusted to new datastructures

- changed test for TALER_denom_blind/TALER_denom_sign_blinded with and
  without TALER_AgeHash

- minor updates and various fixes
2022-01-21 15:41:02 +01:00
Christian Grothoff
e7aeec04f4
The current recoup API is broken. I guess this is another example where "trivial" API changes turn out to have (multiple!) unexpected consequences.
The current "/recoup" API does not have clear idempotency semantics, as we've discussed on the phone.  This is already bad by itself, as it makes it hard to write down what the API does other than "whatever the implementation does".

However, it actually breaks correctness in this (admittedly kinda contrived, but not impossible) case:

Say that we have a coin A obtained via withdrawal and a coin B obtained via refreshing coin A. Now the denominations of A gets revoked..

The wallet does a recoup of A for EUR:1.

Now the denomination of B also gets revoked.  The wallet recoups B (incidentally also for EUR:1) and now A can be recouped again for EUR:1.  But now the exchange is in a state where it will refuse a legitimate recoup request for A because the detection for an idempotent request kicks in.

This is IMHO bad API design, and the exchange should simply always recoup the maximum amount.

Furthermore, we usually follow the principle of "API calls that take up DB space are paid".  With the current recoup API, I can do many tiny recoup requests which the exchange then has to store, right?

I guess it would not be a big change to remove the "amount" value from the recoup/recoup-refresh request bodies, right?

- Florian
2022-01-11 12:47:35 +01:00
Christian Grothoff
aaaaa9a103
fix amount denormalization issue 2022-01-10 09:04:09 +01:00