Installation Information For RIPE Whois Server (RIP)
****************************************************


PREREQUISITES
-------------

These prerequisites are essential for the installation of the RIPE Whois
Server (RIP) software.  Please make sure they are fulfilled before going
on with the installation procedure.

You can get these either from the URL provided, or from the RIPE FTP
site at ftp://ftp.ripe.net/ripe/dbase/software/prereq/

You need to have the following software installed into your system (the
versions that we have used are in brackets).  

        - gcc (2.95.2)
                gcc home page http://www.gnu.org/software/gcc/gcc.html

        - GNU Make (3.79.1)
                home page http://www.gnu.org/software/make/make.html

        - glib (2.0)
                GTK+ home page http://www.gtk.org/

        - libxml2 (2.4.26)
                XML parser library
                Home page http://xmlsoft.org/

        - libxslt (1.0.22)
                XSLT library
                Home page http://xmlsoft.org/XSLT/

        - MySQL Server + Client libraries (3.23.54)
                home page http://www.mysql.com/
                  If installing from source, be sure to specify
                  "--enable-thread-safe-client" when you build the 
                  distribution. You also need to specify "--with-innodb"
                  if you intend to use InnoDB. Please also read the
                  file doc/mysql-fine-tuning.txt to setup mysql to work
                  properly.
  
        - The c-client library, from the IMAP distribution (imap-2002c).
          It is needed for MIME parsing in dbupdate.
                get it at ftp://ftp.cac.washington.edu/imap/old/

        - GNU Privacy Guard (1.0.4 to 1.2.2). 
                home page http://www.gnupg.org/

        - Perl (5.6.0).  It is needed for the ripe2rpsl
          converter, remadmin.pl remote administration interface and
          reverse dns checking extensions.
                home page http://www.perl.com/
          For remadmin.pl, you will also need the Net::Telnet module.
                get it at http://www.cpan.org/modules/by-module/Net/

        - A Java compiler and runtime.  We have used Sun's JDK 1.4.2 for this 
          release.
                home page http://java.sun.com/

        - There are some header files in certain modules
          which are generated by m4. If you want to change them, you
          need to have GNU m4 (1.4) installed in your system.
                home page http://www.seindal.dk/rene/gnu/

        - Some source files in the RPSL parser are generated with Lex and 
          Yacc templates. You will need flex (2.5.4) and GNU bison (1.28)
          if you want to change them.  Their home pages are at:
                http://www.gnu.org/software/flex/flex.html
                http://www.gnu.org/software/bison/bison.html



INSTALLING THE SOFTWARE
-----------------------
To run the database you need to perform the following steps:

0. Download the server software release and unpack it.
   If you are reading this file, you probably have already done this.:-)

1. Compile the code.
      a. Please check the "Prerequisites"; you will not be able to
         compile the code if these are not fulfilled.
      b. Edit the file "build.sh", which is a wrapper to "configure",
         and set your site-specific variables.
         You can also choose your compiler and change the compiler flags
         if necessary.
      c. Issue "./build.sh --prefix=<your_prefix>" in the root directory
         of the tree. <your_prefix> specifies the directory where the
         software will be installed. The suggested default is
         /usr/local/whoisd.
         You can supply any other argument that "configure" would
         accept. 

         Note to FreeBSD users:  You need library that supports
         gethostbyname_r(), the multithreaded DNS lookup call.  The
         BIND package provides this. The CFLAGS variable instructs the
         compiler to use the library. The following is included in build.sh
         script for this purpose.

         CFLAGS="-I/usr/local/bind/include -L/usr/local/bind/lib" 
         export CFLAGS

         Also, you should install DES encryption if you want to support
         dbupdate using crypted passwords.
      d. Issue "make". This will compile all the modules, then whois_rip,
         loader, dbupdate and utilities.
      e. Issue "make install". All the relevant files will be installed
         in <your_prefix>.
      f. Optionally, if you would like to generate the source documentation
         issue "make docs". This will run cxref, gtags and htags programs to
         generate HTML files that you can use to browse the source code
         using your favorite HTML browser. See GNU Global home page 
         (http://www.gnu.org/software/global/ ) for gtags and htags programs
         and see Cxref home page (http://www.gedanken.demon.co.uk/cxref/ )for 
         cxref program.
        


DATABASE CREATION AND LOADING
-----------------------------
HOW IT WORKS


The database creation and loading process is controlled by the script
<your_prefix>/scripts/make_db. 

One of the parameters you need to specify to the make_db script is
whether the database is loaded in one pass or in two passes.

The difference between one-pass and two-pass loads is that in the second
case only primary keys are loaded into the database during the first
pass. An assumption is made that all objects have unique primary keys.
The first phase requires a reduced set of the database indexes making
the overall loading process faster. During the second phase the cross
references between the objects are resolved and stored in the database.

The one-pass loading performs operations similar to a normal server
updates: it is checked if the object already exists in the database, and
all references are resolved. Full set of the database indexes is
involved which makes the loading slower.

The make_db script sets all needed variables and then calls utilities
that actually create the SQL database, load data into it along with the
creation of necessary indexes.

The usage of make_db is:

Usage: make_db -c <config_file> -s <source> -[0|1|2]

where

<config_file> is the name of the whois_rip configuration file
<source> is the source (SOURCE) to be loaded
[0|1|2] number of load passes
0 - just to create an empty database; if the database already exists it 
    will be emptied
1 - one pass. This is a recommended option. May be a little slower than
    2-pass loading but more reliable.
2 - for loading huge databases (more than 1,000,000 objects). High
    requirements on the consistency of the flat files: duplicate objects 
    (objects with the same primary key) will prevent loading.


BEFORE YOU START

A sample object file is included with the distribution. You can load it
into a database by running the make_db script. Make the following steps
to do this.

1. Make sure that you have a MySQL user with all permissions configured
in your MySQL DB. If not you can use the
<your_prefix>/utils/make_mysql_user
script included in your distribution.

2. Edit the configuration files. The most important variables to
configure are definition of the RIPADMIN database and the entries 
in the source configuration file. Sample configuration files installed
in <your_prefix>/conf are partly customized according to your setup.

2.1. In <your_prefix>/conf/rip.config.sample:
2.1.1. Definition of administrative database. This database is used for 
accounting and access control. The format of the record is:

RIPADMIN <sqlhost>,<sqlport>,<dbuser>,<dbpsw>,RIPADMIN

2.1.2. Definition for dbupdate (update front-end). For each source that
you plan to serve submissions to you need one record of this type. The
format is:

UPDSOURCE <source> <db_details> <whois_host>,<whois_port>,<update_port>

where <db_details> specify the MySQL database where dbupdate stores data
for internal use. We recommend to specify the same database as for the
<source> in sources.config.sample file. The format is:

<sqlhost>,<sqlport>,<dbuser>,<dbpsw>,<dbname>

2.2. In <your_prefix>/conf/sources.config.sample you need to define the
database for each source you plan to serve (update, static or NRTM). To get
started just customize the definition for SAMPLE database. You don't need
to create the database, it will be done automatically by make_db script.

Note: If make_db script reports an error that the database already exists,
that means that the database has been created before. The error may be 
ignored.


DATABASE        SAMPLEDB
host:           sqlhost
port:           3306
trx_support:    n
user:           dbuser
password:       dbpsw
/DATABASE

Depending on the desired operation mode you need to define the opMode
entry for each SOURCE in the source configuration file.

Its most common values are:
0 - static, no update thread
2 - update server (serving requests from dbupdate)
4 - NRTM client

3. As soon as this is done you may run the <your_prefix>/make_db script,
which will perform the following operations:

- create main SAMPLE and administrative (RIPADMIN) databases;
- load the database (in one or two passes);
- create the necessary indexes.

cd <your_prefix>/scripts
./make_db -c <your_prefix>/conf/rip.config.sample -s SAMPLE -1 

Note: using -0 option will create an empty database; -1 will load the
database in a single pass, and -2 will do this in 2 passes. For the sample object
file you may try any of these options.

As a result you should have 22 objects loaded in the SAMPLE database
without errors.


Please note that if you are loading the whole RIPE Database, doing this
might take a fair amount of time, even several days. The more powerful
your machine is, the less it will take to do this task.



STARTING THE SERVICE
--------------------

The name of the main Whoisd server is ${prefix}/bin/whois_rip. 

There are 2 wrappers included in the distribution:

<your_prefix>/bin/whoisd_start 
This is a wrapper that restarts the server in case it crashes. Every time 
the server is restarted in the safe (STATIC) mode, which means that no updates 
are processed and the back-end database is accessed in read-only mode.

<your_prefix>/scripts/whoisd.server start|stop
This script starts and shuts down the server using <your_prefix>/bin/whoisd_start
wrapper.

1. Before you use these scripts we would suggest you to start the server alone
from the command line to verify that all basic components are installed correctly.

You need to setup and export the LD_LIBRARY_PATH before. LD_LIBRARY_PATH should 
be set to  "$GLIBLIB:$MYSQLLIB" (where $GLIBLIB and $MYSQLLIB are your
site-specific directories for Glib and MySQL libraries). You may pick up the
value of it in <your_prefix>/bin/whoisd_start script.

$ LD_LIBRARY_PATH=<your_library_path>; export LD_LIBRARY_PATH
$ cd <your_prefix>/bin
$ ./whois_rip -c <your_prefix>/conf/rip.config.sample -p whoisd.pid
Configuration [<your_prefix>/conf/rip.config.sample]:
Check the DB
Checking the Database [SAMPLEDB]...[OK]
  STATUS: The Database is clean 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Source [SAMPLE] Mode UPDATE

That means that the server has been started successfully. You may query it 
at port 43001 (as defined in rip.config.sample).

2. Now you need to customize your <your_prefix>/scripts/whoisd.server

Configuration file to be used located in <your_prefix>/conf directory
CONFIG=rip.config.sample

E-mail address to which notifications about the server crashes will be sent
NOTIFY=myaddress@myhost.mydomain

Number of crashes per day before the restarting loop stops
CRASHES=10

Name of the file where the server PID is stored
pid_file=$BASEDIR/bin/whois_rip.pid

3. You are ready to start the server.

Go to <your_prefix>/utils and run whoisd.server:

$ ./whoisd.server start

The go to <your_prefix>/bin and check whois_rip.err.log file to see that the server has 
started without problems. The server is ready to process updates and answer queries.

4. Stopping the server

$ ./whoisd.server stop



RUNNING DBUPDATE
----------------

The "<your_prefix>/bin/dbupdate" program is used to create/delete/update
objects.

Usage of dbupdate:

There is a long version of each command-line option.

dbupdate -c <config_file> [-m] [-p] [-r] [-n] [-t] [-d] [-o <string>] \
            [-k <string>] [-f <full_path_filename>]

-m --mail: Read from a mail message. This flag will activate the MIME parser.

-f --file: Read input from the given file. If not given, the input is read from
           STDIN.

-c --config: to change the configuration file to be used. It is a mandatory
           flag.

-d --debug: Sets the debug trace mode for this run of dbupdate.

-t --test: Sets dbupdate into test mode, where the as-block, mntner
           and irt objects can be created. Corresponds to the "TESTMODE"
           variable in the config file. Command-line option overrides the
           configuration.

-r --redirect: Send all notifications and acknowledgements to DEFMAIL that is
           defined in the config. For debugging purposes.
      
-p --print: to print out the acknowledgement to stout even when using
          '-m'. Useful for debugging and integration with other programs/
          scripts.

-o --origin: Origin IP addresss string passed from webupdates/syncupdates.

-k --keywords: String containing keywords.

-x --x509: x509 certificate file that the object was signed with.

-X --cert: x509 certificate itself on the command line.

There are several variables that you can/should customize in the
configuration file.

UPDSOURCE: dbupdate now supports updating multiple sources; you
can have, in the config file of dbupdate, more than one "UPDSOURCE"
line with the names of the sources you want to update and some relevant
information. The description of the UPDSOURCE variable can be found in
the main config file.

You can also modify many other variables in the config file. Some of the
most important are:
LOCKDIR
OVERRIDECRYPTEDPW
TMPDIR
MAILCMD
HUMAILBOX
AUTOBOX
PGPPATH
GPGCMD

Before running dbupdate you need to allow access for dbupdate client to
whois_rip update port. Perform the following steps to do this:

1. Update the admin database as follows (admin database is the database
mentioned in the configuration file as RIPADMIN):

 mysql>use RIPADMIN

2. Look at the aaa table:

 mysql>select * from aaa;

3. Add the IP number; in the example below, we add the localhost, 127.0.0.1:

 mysql>insert aaa set prefix=inet_aton('127.0.0.1'), prefix_length=32, source='SAMPLE',
       ripupdate=1, comment='access for dbupdate';


NEAR-REAL-TIME MIRRORS
----------------------

The Whoisd server can only mirror a RPSL stream. If you want to mirror a
RIPE-181 database, or any other non-RPSL database, you need to run 
a "mirror reflector".

This is a tiny program, provided in the distribution under
<your_prefix>/utils/mr, which connects to a specified NRTM server (-h host
-p port) and accepts requests from a RPSL NRTM client (-l listen_port). It
can obviously run in the NRTM client host itself.

After receiving a request, the mirror reflector converts the data stream
on the fly using a specified converter (-f converter); in the case of
RIPE181 to RPSL, you can use the "ripe2rpsl" perl script included in the 
<your_prefix>/scripts directory.

Your Whoisd server needs then to connect to the reflector instead of the real
server, so that it gets a converted RPSL stream.


LOGFILES
--------

During the installation procedure all logfiles are defined in the
configuration file and all needed directories are created.

The log directories for different server subsystems are located under the
<your_prefix>/log directory. The prefixes of filenames are defined in the main
configuration file by corresponding variables. The actual name of the
logfile is composed from a prefix and the date separated by a period (".").

The most important logfiles are (defined by the variables below)
QRYLOG - the query log
ALLRIPERR - the main error log
RIPUPDLOG - the log for all update transactions for a given source



INFRASTRUCTURE
--------------

As part of the installation procedure the following directory layout is
created:

<your_prefix>
 |
 bin/  - main server executables
 conf/ - configuration files 
 log/  - log infrastructure
 scripts/ - bootstrapping and auxillary scripts
  |
  SQL/ - SQL database schema definitions and creation scripts
 utils/ - utilities


This package comes with some infrastructure tools that can be used to
maintain the database. They are not compiled nor installed automatically.
Up to now, the following tools are available:

  - Remote Administration Interface: you can find it under
    utils/remadmin.pl . This script connects to the configuration port
    of your server to perform administrative tasks such as pausing and
    restarting updates, checking the status of the threads, etc. It is
    actually a non-interactive interface to 
    "telnet <server> <configport>".
    Currently, you need PERL and the module Net::Telnet to use this
    interface. It will be eventually rewritten in C.

  - Text Export: This utility builds plain text files, such as the ones
    used for loading, from a v3.0 Whois server. For more details,
    please check the file README in the software distribution in 
    utils/text_export .

  - Mirror Reflector: This utility is located in <your_prefix>/utils/mr
    It is used to convert or filter the NRTM streams that are
    not compliant with RPSL. Example of such stream is RIPE-181 one.

  - make_mysql_user: This utility creates a MySQL user for DB access.
