update manual a bit more

This commit is contained in:
Christian Grothoff 2017-03-20 16:46:13 +01:00
parent e72d0ec3ff
commit 6050dad63e
No known key found for this signature in database
GPG Key ID: 939E6BE1E29FC3CC

View File

@ -74,6 +74,159 @@ Indices
@node Introduction @node Introduction
@chapter Introduction @chapter Introduction
This manual is an early draft that still needs significant editing
work to become readable.
@section About GNU Taler
GNU Taler is an open protocol for an electronic payment system with a
free software reference implementation. GNU Taler offers secure, fast
and easy payment processing using well understood cryptographic
techniques. GNU Taler allows customers to remain anonymous, while
ensuring that merchants can be held accountable by governments.
Hence, GNU Taler is compatible with anti-money-laundering (AML) and
know-your-customer (KYC) regulation, as well as data protection
regulation (such as GDPR).
GNU Taler is not yet production-ready, after following this manual
you will have a backend that can process payments in ``KUDOS'', but
not regular currencies. This is not so much because of limitations
in the backend, but because we are not aware of a Taler exchange
operator offering regular currencies today.
@section About this manual
This tutorial targets system administrators who want to install and
operate a GNU Taler exchange.
@section Organizational prerequisites
Operating a GNU Taler exchange means that you are operating a payment
service provider, which means that you will most likely need a bank
license and/or follow applicable financial regulation.
\cindex{availability}
\cindex{backup}
\cindex{replication}
GNU Taler payment service providers generally need to ensure high
availability and have @emph{really} good backups (synchronous
replication, asynchronous remote replication, off-site backup, 24/7
monitoring, etc.).@footnote{Naturally, you could operate a Taler
exchange for a toy currency without any real value on low-cost setups
like a Raspberry Pi, but we urge you to limit the use of such setups
to research and education as with GNU Taler data loss instantly
results in financial losses.} This manual will not cover these
aspects of operating a payment service provider.
\cindex{HSM}
\cindex{offline}
\cindex{database}
\cindex{operational security}
We will assume that you can operate a (high-availability,
high-assurance) Postgres database. Furthermore, we expect some
moderate familiarity with the compilation and installation of free
software packages. You need to understand the cryptographic concepts
of private and public keys and must be able to protect private keys
stored in files on disk. An exchange uses an @emph{offline} master
key as well as @emph{online} keys. You are advised to secure your
private master key and any copies on encrypted, always-offline
computers. Again, we assume that you are familiar with good best
practices in operational security, including securing key
material.@footnote{The current implementation does not make provisions
for secret splitting. Still, the use of a hardware security module
(HSM) for protecting private keys is adviseable, so please contact the
developers for HSM integration support.}
@section Architecture overview
@cindex crypto-currency
@cindex bank
@cindex escrow
@cindex coin
Taler is a pure payment system, not a new crypto-currency. As such, it
operates in a traditional banking context. In particular, this means
that in order to receive funds via Taler, the merchant must have a
regular bank account, and payments can be executed in ordinary
currencies such as USD or EUR. Similarly, the Taler exchange must
interact with a bank. The bank of the exchange holds the exchange's
funds in an escrow account.
@cindex reserve
@cindex fee
@cindex aggregator
@cindex deposit
When customers wire money to the escrow account, the bank notifies
the exchange about the incoming wire transfers. The exchange then
creates a @emph{reserve} based on the subject of the wire transfer.
The wallet which knows the secret key matching the wire transfer
subject can then withdraw coins from the reserve, thereby draining
it. The liability of the exchange against the reserve is thereby
converted into a liability against digital coins issued by the
exchange. When the customer later spends the coins at a merchant,
and the merchant @emph{deposits} the coins at the exchange, the
exchange first @emph{aggregates} the amount from multiple deposits
from the same merchant and then instructs its bank to make a
wire transfer to the merchant, thereby fulfilling its obligation
and eliminating the liability. The exchange charges @emph{fees}
for some or all of its operations to cover costs and possibly make
a profit.
@cindex auditor
@cindex accounting
@emph{Auditors} are third parties, for example financial regulators,
that verify that the exchange operates correctly. The same software
is also used to calculate the exchange's profits, risk and liabilities
by the accountants of the exchange.
The Taler software stack for an exchange consists of the
following components:
@itemize
@cindex HTTP frontend
@item The HTTP frontend interacts with Taler wallets and
merchant backends. It is used to withdraw coins, deposit
coins, refresh coins, issue refunds, map wire transfers to
Taler transactions, inquire about the exchange's bank account
details, signing keys and fee structure.
The binary is the @code{taler-exchange-httpd}.
@cindex Aggregator
@item The aggregator combines multiple deposits made by
the same merchant and (eventually) triggers wire transfers for the
aggregate amount. The merchant can control how quickly wire
transfers are made. The exchange may be charge a fee per wire transfer
to discourage excessively frequent transfers. The binary
is the @code{taler-exchange-aggregator}.
@cindex Auditor
@item The auditor verifies that the transactions performed by
the exchange were done properly. It checks the various signatures,
totals up the amounts and alerts the operator to any inconsistencies.
It also computes the expected bank balance, revenue and risk exposure
of the exchange operator. The main binary is the
@code{taler-auditor}.
@cindex Wire plugin
@item A wire plugin enables the HTTP frontend to talk to the
bank. Its role is to allow the exchange to validate bank
addresses (i.e. IBAN numbers), for the aggregator to execute
wire transfers and for the auditor to query bank transaction
histories. Wire plugins are @emph{plugins} as there can be
many different implementations to deal with different
banking standards. Wire plugins are automatically located
and used by the exchange, aggregator and auditor.
@cindex DBMS
@cindex Postgres
@item The exchange requires a DBMS to stores the transaction history for
the Taler exchange and aggregator, and a (typically separate) DBMS for
the Taler auditor.
For now, the GNU Taler reference implemenation only supports Postgres,
but the code could be easily extended to support another DBMS.
@end itemize
@c The following image illustrates the various interactions of these
@c key components:
@c @center @image{arch, 3in, 4in}
@node Installation @node Installation
@ -105,7 +258,7 @@ GNU libunistring >= 0.9.3
libcurl >= 7.26 (or libgnurl >= 7.26) libcurl >= 7.26 (or libgnurl >= 7.26)
@item @item
GNU libmicrohttpd >= 0.9.39 GNU libmicrohttpd >= 0.9.52
@item @item
GNU libgcrypt >= 1.6 GNU libgcrypt >= 1.6
@ -114,7 +267,7 @@ GNU libgcrypt >= 1.6
libjansson >= 2.7 libjansson >= 2.7
@item @item
Postgres >= 9.4, including libpq Postgres >= 9.6, including libpq
@item @item
libgnunetutil (from Git) libgnunetutil (from Git)
@ -176,7 +329,11 @@ if you installed GNUnet to @code{/usr/local} in the previous step.
@node Configuration @node Configuration
@chapter Configuration @chapter Configuration
In this document, we assume that @code{$HOME/.config/taler.conf} is being customized. @c In this document, we assume that @code{$HOME/.config/taler.conf} is being customized.
This chapter provides an overview of the exchange configuration. Or
at least eventually will do so, for now it is a somewhat wild
description of some of the options.
@menu @menu
* Keying:: * Keying::
@ -218,7 +375,8 @@ The exchange works with three types of keys:
@cite{master_public_key}: Must specify the exchange's master public key. @cite{master_public_key}: Must specify the exchange's master public key.
@end itemize @end itemize
@cite{sign keys}: the following two options under @cite{[exchange_keys]} section control @cite{sign keys}: @cite{sign keys}: the following two options under
@cite{[exchange_keys]} section control @cite{sign keys}:
@itemize @itemize
@ -227,7 +385,11 @@ The exchange works with three types of keys:
@cite{signkey_duration}: How long should one signing key be used? @cite{signkey_duration}: How long should one signing key be used?
@item @item
@cite{lookahead_sign}: How much time we want to cover with our @cite{signkeys}? Note that if @cite{signkey_duration} is bigger than @cite{lookahead_sign}, @cite{taler-exchange-keyup} will generate a quantity of @cite{signkeys} which is sufficient to cover all the gap. See keys-duration. @cite{lookahead_sign}: How much time we want to cover with our
@cite{signkeys}? Note that if @cite{signkey_duration} is bigger than
@cite{lookahead_sign}, @cite{taler-exchange-keyup} will generate a
quantity of @cite{signkeys} which is sufficient to cover all the
gap. See keys-duration.
@end itemize @end itemize
@ -235,26 +397,32 @@ The exchange works with three types of keys:
@section Serving @section Serving
The exchange can serve HTTP over both TCP and UNIX domain socket. It needs this The exchange can serve HTTP over both TCP and UNIX domain socket. It
configuration @emph{twice}, because it opens one connection for ordinary REST calls, and one needs this configuration @emph{twice}, because it opens one connection
for "/admin" and "/test" REST calls, because the operator may want to restrict the access to "/admin". for ordinary REST calls, and one for "/admin" and "/test" REST calls,
because the operator may want to restrict the access to "/admin".
The following values are to be configured under the section @cite{[exchange]} and @cite{[exchange-admin]}: The following values are to be configured under the section
@cite{[exchange]} and @cite{[exchange-admin]}:
@itemize @itemize
@item @item
@cite{serve}: must be set to @cite{tcp} to serve HTTP over TCP, or @cite{unix} to serve HTTP over a UNIX domain socket @cite{serve}: must be set to @cite{tcp} to serve HTTP over TCP, or
@cite{unix} to serve HTTP over a UNIX domain socket
@item @item
@cite{port}: Set to the TCP port to listen on if @cite{serve} Is @cite{tcp}. @cite{port}: Set to the TCP port to listen on if @cite{serve} Is
@cite{tcp}.
@item @item
@cite{unixpath}: set to the UNIX domain socket path to listen on if @cite{serve} Is @cite{unix} @cite{unixpath}: set to the UNIX domain socket path to listen on if
@cite{serve} Is @cite{unix}
@item @item
@cite{unixpath_mode}: number giving the mode with the access permissiON MASK for the @cite{unixpath} (i.e. 660 = rw-rw----). @cite{unixpath_mode}: number giving the mode with the access
permissiON MASK for the @cite{unixpath} (i.e. 660 = rw-rw----).
@end itemize @end itemize
The exchange can be started with the @cite{-D} option to disable the administrative The exchange can be started with the @cite{-D} option to disable the administrative
@ -283,9 +451,11 @@ option @cite{currency} in section @cite{[taler]}.
@subsection Wireformat @subsection Wireformat
The wireformat is the protocol to be used between the exchange and the banks. The wireformat is the protocol to be used between the exchange and the
The option is @cite{wireformat}, under section @cite{[exchange]}. The exchange currently supports banks. The option is @cite{wireformat}, under section
the @cite{test} wireformat. This wireformat is used for testing the system against a fictional bank. @cite{[exchange]}. The exchange currently supports the @cite{test}
wireformat. This wireformat is used for testing the system against a
fictional bank.
@cartouche @cartouche
@quotation Note @quotation Note
@ -297,17 +467,20 @@ The SEPA wireformat is work in progress.
@subsection Incoming @subsection Incoming
The bank account where the exchange gets money from customers is configured under The bank account where the exchange gets money from customers is
the section @cite{[exchange-wire-incoming-X]}, where @cite{X} matches the value given to the configured under the section @cite{[exchange-wire-incoming-X]}, where
option @cite{wireformat}. This section contains only one option: @cite{X_response_file}, which @cite{X} matches the value given to the option @cite{wireformat}. This
takes the path to a text file containing the exchange's bank account details in JSON section contains only one option: @cite{X_response_file}, which takes
format. the path to a text file containing the exchange's bank account details
in JSON format.
The command line tool @cite{taler-exchange-wire} is used to create such a file. The command line tool @cite{taler-exchange-wire} is used to create such a file.
For example, the utility may be invoked as follows: For example, the utility may be invoked as follows:
@example @example
$ taler-exchange-wire -j '@{"name": "The Exchange", "account_number": 10, "bank_uri": "https://bank.demo.taler.net", "type": "test"@}' -t test -o exchange.json $ taler-exchange-wire -j '@{"name": "The Exchange", "account_number":
10, "bank_uri": "https://bank.demo.taler.net", "type": "test"@}' -t
test -o exchange.json
@end example @end example
Note that the value given to option @cite{-t} must match the value in the JSON's field @code{"type"}. Note that the value given to option @cite{-t} must match the value in the JSON's field @code{"type"}.
@ -319,11 +492,12 @@ requests.
@node Outgoing @node Outgoing
@subsection Outgoing @subsection Outgoing
This exchange's bank account is used to give money to merchants, after
This exchange's bank account is used to give money to merchants, after successful successful
deposits@footnote{https://api.taler.net/api-exchange.html#deposit-par} deposits@footnote{https://api.taler.net/api-exchange.html#deposit-par}
operations. If @cite{test} is the chosen wireformat, the outcoming bank account is configured by the following operations. If @cite{test} is the chosen wireformat, the outcoming
options under @cite{[exchange-wire-outcoming-test]}: bank account is configured by the following options under
@cite{[exchange-wire-outcoming-test]}:
@quotation @quotation
@ -348,7 +522,7 @@ measure, may want to instruct the bank that the incoming bank account is only su
@node Database,Coins denomination keys,Bank account,Configuration @node Database
@section Database @section Database
@ -377,59 +551,84 @@ db = postgres
db_conn_str = postgres:///talerdemo db_conn_str = postgres:///talerdemo
@end example @end example
@node Coins denomination keys,Keys duration,Database,Configuration @node Coins denomination keys
@section Coins (denomination keys) @section Coins (denomination keys)
Sections specifying denomination (coin) information start with "coin_". By convention, the name continues with "$CURRENCY_[$SUBUNIT]_$VALUE", i.e. @cite{[coin_eur_ct_10]} for a 10 cent piece. However, only the "coin_" prefix is mandatory. Each "coin_"-section must then have the following options: Sections specifying denomination (coin) information start with
"coin_". By convention, the name continues with
"$CURRENCY_[$SUBUNIT]_$VALUE", i.e. @cite{[coin_eur_ct_10]} for a 10
cent piece. However, only the "coin_" prefix is mandatory. Each
"coin_"-section must then have the following options:
@itemize @itemize
@item @item
@cite{value}: How much is the coin worth, the format is CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is "EUR:0.10". @cite{value}: How much is the coin worth, the format is
CURRENCY:VALUE.FRACTION. For example, a 10 cent piece is "EUR:0.10".
@item @item
@cite{duration_withdraw}: How long can a coin of this type be withdrawn? This limits the losses incurred by the exchange when a denomination key is compromised. @cite{duration_withdraw}: How long can a coin of this type be
withdrawn? This limits the losses incurred by the exchange when a
denomination key is compromised.
@item @item
@cite{duration_overlap}: What is the overlap of the withdrawal timespan for this coin type? @cite{duration_overlap}: What is the overlap of the withdrawal
timespan for this coin type?
@item @item
@cite{duration_spend}: How long is a coin of the given type valid? Smaller values result in lower storage costs for the exchange. @cite{duration_spend}: How long is a coin of the given type valid?
Smaller values result in lower storage costs for the exchange.
@item @item
@cite{fee_withdraw}: What does it cost to withdraw this coin? Specified using the same format as @cite{value}. @cite{fee_withdraw}: What does it cost to withdraw this coin?
Specified using the same format as @cite{value}.
@item @item
@cite{fee_deposit}: What does it cost to deposit this coin? Specified using the same format as @cite{value}. @cite{fee_deposit}: What does it cost to deposit this coin? Specified
using the same format as @cite{value}.
@item @item
@cite{fee_refresh}: What does it cost to refresh this coin? Specified using the same format as @cite{value}. @cite{fee_refresh}: What does it cost to refresh this coin? Specified
using the same format as @cite{value}.
@item @item
@cite{rsa_keysize}: How many bits should the RSA modulus (product of the two primes) have for this type of coin. @cite{rsa_keysize}: How many bits should the RSA modulus (product of
the two primes) have for this type of coin.
@end itemize @end itemize
@node Keys duration,,Coins denomination keys,Configuration
@node Keys duration
@section Keys duration @section Keys duration
Both @cite{signkeys} and @cite{denom keys} have a starting date. The option @cite{lookahead_provide}, under section @cite{[exchange_keys]}, is such that only keys Both @cite{signkeys} and @cite{denom keys} have a starting date. The
whose starting date is younger than @cite{lookahead_provide} will be issued by the exchange. option @cite{lookahead_provide}, under section @cite{[exchange_keys]},
is such that only keys whose starting date is younger than
@cite{lookahead_provide} will be issued by the exchange.
@cite{signkeys}. The option @cite{lookahead_sign} is such that, being @cite{t} the time when @cite{taler-exchange-keyup} @cite{signkeys}. The option @cite{lookahead_sign} is such that, being
is run, @cite{taler-exchange-keyup} will generate @cite{n} @cite{signkeys}, where @cite{t + (n * signkey_duration) = t + lookahead_sign}. In other words, we generate a number of keys which is sufficient to cover a period of @cite{t} the time when @cite{taler-exchange-keyup} is run,
@cite{lookahead_sign}. As for the starting date, the first generated key will get a starting time of @cite{t}, @cite{taler-exchange-keyup} will generate @cite{n} @cite{signkeys},
and the @cite{j}-th key will get a starting time of @cite{x + signkey_duration}, where @cite{x} is the starting time where @cite{t + (n * signkey_duration) = t + lookahead_sign}. In other
of the @cite{(j-1)}-th key. words, we generate a number of keys which is sufficient to cover a
period of @cite{lookahead_sign}. As for the starting date, the first
generated key will get a starting time of @cite{t}, and the
@cite{j}-th key will get a starting time of @cite{x +
signkey_duration}, where @cite{x} is the starting time of the
@cite{(j-1)}-th key.
@cite{denom keys}. The option @cite{lookahead_sign} is such that, being @cite{t} the time when @cite{taler-exchange-keyup} @cite{denom keys}. The option @cite{lookahead_sign} is such that,
is run, @cite{taler-exchange-keyup} will generate @cite{n} @cite{denom keys} for each denomination, where being @cite{t} the time when @cite{taler-exchange-keyup} is run,
@cite{t + (n * duration_withdraw) = t + lookahead_sign}. In other words, for each denomination, we generate a @cite{taler-exchange-keyup} will generate @cite{n} @cite{denom keys}
number of keys which is sufficient to cover a period of @cite{lookahead_sign}. As for the starting date, the for each denomination, where @cite{t + (n * duration_withdraw) = t +
first generated key will get a starting time of @cite{t}, and the @cite{j}-th key will get a starting time of lookahead_sign}. In other words, for each denomination, we generate a
@cite{x + duration_withdraw}, where @cite{x} is the starting time of the @cite{(j-1)}-th key. number of keys which is sufficient to cover a period of
@cite{lookahead_sign}. As for the starting date, the first generated
key will get a starting time of @cite{t}, and the @cite{j}-th key will
get a starting time of @cite{x + duration_withdraw}, where @cite{x} is
the starting time of the @cite{(j-1)}-th key.
@ -438,88 +637,12 @@ first generated key will get a starting time of @cite{t}, and the @cite{j}-th ke
@menu @menu
* Deploying to stable::
* Database upgrades::
* Standalone deployment:: * Standalone deployment::
* Database upgrades::
* Deployment on demo.taler.net::
@end menu @end menu
@node Deploying to stable
@section Deploying to stable
First, make sure that the deployment @emph{AND} the deployment scripts work on the @cite{test.taler.net} deployment.
For all repositories that have a separate stable branch (currently exchange.git,
merchant.git, merchant-frontends.git, bank.git, landing.git) do:
@example
$ cd $REPO
$ git pull origin master stable
$ git checkout stable
# option a: resolve conflicts resulting from hotfixes
$ git merge master
$ ...
# option b: force stable to master
$ git update-ref refs/heads/stable master
$ git push # possibly with --force
# continue development
$ git checkout master
@end example
Log into taler.net with the account that is @emph{not} active by looking
at the @cite{sockets} symlink of the @cite{demo} account.
The following instructions wipe out the old deployment completely.
@example
$ ls -l ~demo/sockets
[...] sockets -> /home/demo-green/sockets/
@end example
In this case, @cite{demo-green} is the active deployment, and @cite{demo-blue} should be updated.
After the update is over, the @cite{/home/demo/sockets} symlink will be pointed to @cite{demo-blue}.
@example
# Remove all existing files
$ find $HOME -exec rm -fr @{@} \;
$ git clone /var/git/deployment.git
$ ./deployment/bootstrap-bluegreen demo
# set environment appropriately
$ . activate
$ taler-deployment-build
# upgrade the database! this
# process depends on the specific version
$ taler-deployment-start
# look at the logs, verify that everything is okay
@end example
Now the symlink can be updated.
@node Database upgrades
@section Database upgrades
The exchange db can be re-initialized with
@example
$ taler-exchange-dbinit -r
@end example
CAUTION: YOU WILL LOSE ALL DATA WITH THIS!
@node Standalone deployment @node Standalone deployment
@section Standalone deployment @section Standalone deployment
@ -649,9 +772,104 @@ at @code{<DEPLOYMENT-REPO>/etc/nginx}.
@end cartouche @end cartouche
@node Database upgrades
@section Database upgrades
Currently, there is no way to upgrade the database between Taler versions.
The exchange database can be re-initialized using:
@example
$ taler-exchange-dbinit -r
@end example
However, running this command will result in all data in the database
being lost, which may result in significant financial liabilities as
the exchange can then not detect double-spending. Hence this
operation must not be performed in a production system.
@node Deployment on demo.taler.net
@section Deployment on demo.taler.net
This section describes how to upgrade the exchange deployment on the
@url{taler.net} Web site. Here, the deployment scripts include a
``stable'' setup at @url{demo.taler.net} and an ``experimental'' setup
at @url{test.taler.net}. This section documents the steps for moving
the ``experimental'' logic to the ``stable'' site. It is mostly
useful for administrators of @url{taler.net}, but given that all of
the configuration files are public, it may also make a good starting
point for others.
First, make sure that the deployment @emph{AND} the deployment scripts work on the @cite{test.taler.net} deployment.
For all repositories that have a separate stable branch (currently exchange.git,
merchant.git, merchant-frontends.git, bank.git, landing.git) do:
@example
$ cd $REPO
$ git pull origin master stable
$ git checkout stable
# option a: resolve conflicts resulting from hotfixes
$ git merge master
$ ...
# option b: force stable to master
$ git update-ref refs/heads/stable master
$ git push # possibly with --force
# continue development
$ git checkout master
@end example
Log into taler.net with the account that is @emph{not} active by looking
at the @cite{sockets} symlink of the @cite{demo} account.
The following instructions wipe out the old deployment completely.
@example
$ ls -l ~demo/sockets
[...] sockets -> /home/demo-green/sockets/
@end example
In this case, @cite{demo-green} is the active deployment, and @cite{demo-blue} should be updated.
After the update is over, the @cite{/home/demo/sockets} symlink will be pointed to @cite{demo-blue}.
@example
# Remove all existing files
$ find $HOME -exec rm -fr @{@} \;
$ git clone /var/git/deployment.git
$ ./deployment/bootstrap-bluegreen demo
# set environment appropriately
$ . activate
$ taler-deployment-build
# upgrade the database! this
# process depends on the specific version
$ taler-deployment-start
# look at the logs, verify that everything is okay
@end example
Now the symlink can be updated.
@node Diagnostics @node Diagnostics
@chapter Diagnostics @chapter Diagnostics
This chapter includes various (very unpolished) sections on specific topics
that might be helpful to understand how the exchange operates, which files
should be backed up. The information may also be helpful for diagnostics.
@menu @menu
* Configuration format:: * Configuration format::
* Reserve management:: * Reserve management::
@ -737,8 +955,10 @@ URL, or a database name.
@section Reserve management @section Reserve management
Incoming transactions to the exchange's provider result in the creation or update of reserves, identified by their reserve key. Incoming transactions to the exchange's provider result in the
The command line tool @cite{taler-exchange-reservemod} allows create and add money to reserves in the exchange's database. creation or update of reserves, identified by their reserve key. The
command line tool @cite{taler-exchange-reservemod} allows create and
add money to reserves in the exchange's database.