scc-srv - introduction SCC-SRV


       scc-srv	 1.17.5


       The complete functionality of the server part of SCC consists of:

       - receive files sent by scc from client systems
       - transfer of these files to the directory used by the web server
       - summarize data in the received files
       - allow search and compare functionality through a web server

       Refer to the manual pages of scc for details concerning the SCC client.

   Summary of transfer of SCC-data
       Basically there are three setups: push, pull and two-step.

       In the push setup, all clients schedule scc with	 the  -p  and  the  -s
       options and transferred data is processed on scc-srv by means of sched-
       uled invocations of  scc-update(1).   This  setup  requires  (non-root)
       access  of  all clients to scc-srv, except when data is transferred via

       In the pull setup, scc-srv schedules scc-pull(1) with the -R option  to
       run the client, obtain the data and process the transferred data.  This
       setup requires  superuser  access  from	scc-srv	 to  all  clients  and
       involves	 all  systems  in a realm and all scc client scripts should be
       accessible via the same path.  Note that scc-win clients do not support
       pulling scc data.

       In  the	two-step  setup,  all clients schedule scc with -p pull option
       without the -s option and scc-srv schedules scc-pull(1) with the -R and
       -n  options to obtain and process the client data.  This setup requires
       (non-root) access from scc-srv to all clients and involves all  systems
       in  a  realm  and all scc client data should be accessible via the same

       These three setups could be mixed on a realm basis.

   Summary of activation
       The following list summarizes the steps to  get	scc-srv	 working.   We
       assume that SCC-data is transferred via email and that it is referenced
       by a symbolic link from the document root of the web server.  When  the
       snapshots are not transported via email, you can skip the corresponding
       steps.  Refer to the remainder of this document	for  a	more  detailed
       description and other setups.

       1  activate the web server and email-processing
       2  install scc-srv
       3  extend the mail-alias file:
	     scc-transfer: "| /opt/scc-srv/bin/scc-receive-mail"
       4  activate the new alias:
       5  determine the user and group used during the email delivery
	  and assign the proper ownership (assume mail:mail):
	     scc-update -m mail:mail
       6  test whether scc-setup is able to determine the proper defaults by means of:
	  optional set environment variables to supply the proper values and activate
	     scc-setup -a
       7  schedule scc-update to process incoming SCC-data, frequency depends
	  upon the amount of data and time of arrival of new SCC-data.
	     scc-update	   # no arguments/options required

       It is also possible to pull the scc-files from the clients after remote
       execution.  This avoids steps 3, 4, 5 and the scheduling of scc on  the
       clients.	  Refer	 to  the  section "Pulling files from the clients" for
       more details.

       First of all we need a working scc server system.  Later we will need a
       client  system  to  send its data through email, scp, ftp, rcp or cp to
       the server.  Install scc-srv  with  your	 package  management  software
       according to the instructions on the download page of the website.

       This installs several programs in the directory /opt/scc-srv/bin , man-
       ual pages in /opt/scc-srv/man , documentation in /opt/scc-srv/doc and a
       directory  tree	under /var/opt/scc-srv/data.  To access these programs
       and manual pages, you can extend your PATH and MANPATH variables during
       the configuration phase.	 During "normal" operations, direct access and
       usage of the scc-server software is not required and the following set-
       tings are not necessary after the completion of the configuration.

	  export PATH MANPATH

       When  you  want	to install the software in another directory, download
       the source tarball, customize and use the script "relocate" and	gener-
       ate  the	 appropriate  depot/package/rpm.   The source tarball contains
       scripts to generate all native install formats.	This can  be  achieved
       by  unpacking  the  source  tarball  and	 adding/modifying the required
       files.  Refer to the README file in the source tree  for	 more  details
       concerning producing the depot/rpm/package.

       The following paragraphs describe how the data-directory tree should be
       used to receive and transfer the files to their ultimate location.

   Receiving files
       When the scc client has collected its data,  it	can  send  it  through
       email,  scp,  ftp,  rcp or cp.  The destination of the files depends on
       the communication program:

	  - cp:	       <directory>
	  - ftp:       srv.dom:<directory>
	  - scp/rcp:   <user>@srv.dom:<directory>
	  - email:     <user>@srv.dom

   Receiving files using scp/rcp
       Suppose you decide to send data from (some of)  your  systems  to  your
       scc-server  by  means of scp or rcp.  Your steps depend on the way file
       transfer between systems in your network is done at the	moment.	  When
       all or most of your systems use scp of rcp for user root, the necessary
       key-files and rhost-files are  already  in  place.   Each  transfer  on
       behalf of scc from a client can use the existing communication path and
       has to specify the full path of the destination directory.

	  scc -p scp \
	      -s root@srv.dom:/var/opt/scc-srv/data/transfer/cp

       When you use several accounts to transfer files to your server, you can
       create  several	subdirectories	in  /var/opt/scc-srv/data/transfer/cp,
       each owned by a specific user.

       Add the proper key files or rhosts file to the home-directory  of  (for
       example)	 user  scc  to enable communication between the client and the
       server.	Clients can now send the data using the following command:

	  /opt/scc/bin/scc -p scp -s scc@srv.dom:

   Receiving files using ftp
       Suppose you decide to send data from (some of) your systems with ftp to
       your  scc-server.   You	have  to create a ~/.netrc file containing the
       data of the account that receives the SCC-data.	The contents  of  this
       file are:

       machine <scc-srv> login <account> password <password>

       Make sure only root can read this file as it contains a plaintext pass-
       word.  This file can also be used with an anonymous ftp-server.	It  is
       obvious	that  you should not use the root-account of the scc-server in
       this file.  Use a separate account as described with the transfer using

   Receiving files using email
       Suppose you decide to send data from (some of) your systems by email to
       your scc-server.	 We have to extend the alias file  on  the  scc-server
       with  an	 alias that triggers a program that puts the email contents in
       the proper directory.  This program is scc-receive-mail(1).  So, extend
       your aliases file with:

	  scc-transfer:	   "| /opt/scc-srv/bin/scc-receive-mail"

       The   default   destination   directory	 of   scc-receive-mail(1)   is
       /var/opt/scc-srv/data/transfer/mail.  Another directory can  be	speci-
       fied  as	 an  argument  for  scc-receive-mail(1).  Do not forget to run
       newaliases to activate the new alias.

       To reduce the likelihood of DOS-attacks, you should use	a  non-obvious
       email-alias and not the alias from this documentation.

       We  have to consider the following security aspects of programs invoked
       through an email alias.	The program scc-receive-mail(1) runs under the
       uid/gid specified by the mail-handler.  On a vanilla RedHat system with
       sendmail, this is  mail/mail.   The  destination	 directory  should  be
       writable	 for this user.	 The rpm for scc-srv sets the uid:gid for this
       directory to root:root.	To change this, use  the  -m  option  of  scc-

	  scc-update -m mail:mail

       Subsequent calls of scc-update(1) do not need to specify this option.

       On  some	 systems the smrsh option (sendmail restricted shell) is acti-
       vated.	This  means  that  only	 programs  located  in	the  directory
       /etc/smrsh  can	be activated through an email alias.  When you use the
       smrsh option,  you  have	 to  add  some	additional  programs  to  this
       restricted shell environment:

	  cd /etc/smrsh
	  ln -s /opt/scc-srv/bin/scc-receive-mail scc-receive-mail
	  ln -s /bin/cat cat

       Modify  the  file  to alter this option.  The program scc-
       receive-mail(1) is not complicated and only uses the program  /bin/cat.
       Now  clients  can  send their data to the scc-server with the following

	  scc -s scc-transfer@srv.dom

       When you use another mail-handler, like postfix, all you need to do  is
       to determine the user:group that is used to run scc-receive-mail(1) and
       change the ownership for the transfer-directory with the -m option.

   Receiving files using cp
       The destination has to be a directory on the client (and server).  This
       option can be used on the server and with NFS.

   Pulling files from the clients
       In  the case of password-less ssh/rsh communication from your server to
       clients, it is also possible to pull the scc-files  from	 the  clients.
       Use  scc-pull(1)	 to  start  scc	 on the clients and pull the resulting
       files to the server.  The program supports several options  controlling
       the  execution  on  the	clients.   It also supports several options to
       specify from which clients to pull the scc-files.  In it simplest form,
       the  program  can  be called without any options.  In that case, scc is
       executed in the background on all systems whose snapshot resides in the
       realm  All.   After waiting for 10 minutes, the scc-files are retrieved
       from all clients and scc-update is called to update the website.	  Con-
       sult the manual page of scc-pull(1) for the details.

       After installing scc on a new client, the system has to be added to the
       command-line of scc-pull.  When scc-pull uses the -R option to  process
       all systems from a realm, it has to be called manually once with the -S
       option to let the files become part of realm All.  Possibly you have to
       use  scc-realm(1) to add the new system to the required realm.  Now the
       scc-files of the new client reside in the required realm and subsequent
       calls  of scc-pull will access the system.  Before using scc-pull for a
       new system, you should test the password-less  communication  from  the
       scc-srv to the new system.

   Combining push and pull
       Schedule scc on the clients and specify the -p pull option.  Now sched-
       ule scc-pull(1) on scc-srv and specify the -n option.  Make  sure  that
       scc-pull(1)  runs  after	 the  last client has finished.	 Otherwise the
       data of running clients are not transferred.

   Transfer of data to web server
       The data, that is sent by the scc-clients, ends up in directories  "cp"
       and  "mail" under /var/opt/scc-srv/data/transfer.  The files are packed
       by tar and compressed by gzip or compress.  Data sent by email is  also
       uuencoded.  The command scc-transfer(1) unpacks the data into files and
       transfers  them	to  the	 directory  /var/opt/scc-srv/data/www/All  and
       assigns	the  proper  permissions  and ownership.  It is called by scc-

       The web server has to read the files.  Therefore we have to change  the
       ownership  of the files to the user that is used to run the web server.
       Use the -w option of scc-update(1)  to  change  the  ownership  of  the
       directories and files.  For example:

	  scc-update -w apache

       Subsequent  calls  of scc-update(1) do not need to specify this option.
       Now all html-files, snapshots, log files and summaries are  present  in
       the directory /var/opt/scc-srv/data/www/All.

       The  summaries  of  the	runs  on the clients are generated by scc-sum-
       mary(1).	 The specification of the System Tools (required by scc-smt(1)
       is  done	 by editing the file /var/opt/scc-srv/data/www/All/custom/scc-
       smt-select Refer to the manual page of  scc-smt(1)  for	more  details.
       The  specification of the rules for scc-rules(1) is done by editing the
       file /var/opt/scc-srv/data/www/All/custom/scc-rules.conf Refer  to  the
       manual page of scc-rules(1) for more details.  These three programs are
       called by scc-update(1).

       Add scc-update(1) to the root-crontab.  The time and  frequency	depend
       upon the time and frequency you use for running SCC on the clients.

   Transfer through a firewall
       When a group of systems is separated from scc-srv by a firewall, one of
       these client systems can act as a gateway for the transfer.  Only  this
       system requires an (ssh) allow rule in the firewall.

       Install	scc-srv	 on the gateway system and ignore the configuration of
       the web-part.  Make sure it contains the data of all of its’ local sys-
       tems.   Either  push data from the client to the gateway system or pull
       data from the clients to the gateway system.  In the latter  case,  use
       the  -U	option of scc-pull to avoid transferring the collected data to
       the web directory of scc-srv on	the  gateway  and  keep	 them  in  the
       data/transfer/cp directory.

       Now make sure that the main scc-srv collects the data of all firewalled
       clients via the gateway system by means of the following command:

	  scc-pull -n -L <gateway_system> -D /var/opt/scc-srv/data/transfer/cp

       As the local clients of the gateway system cannot be accessed directly,
       the -R option of scc-pull cannot be used unless all directly accessible
       systems are in a separate, dedicated realm.

   Performance considerations
       We tested scc-update with many snapshots and  scc-rules	consumes  some
       40%  of	the  time  scc-update  requires.   When you do not require the
       checks of  the  rules,  remove  the  configuration  file	 /var/opt/scc-
       srv/data/www/All/custom/scc-rules.conf to speed up scc-update.  The web
       interface notices the absence of this file and does not show the corre-
       sponding option.

       The  programs  have  been tested with 100, 1000 and 2000 snapshots.  In
       that range, there is a linear dependency between the required time  and
       the number of systems.

   Display of SCC-data by the web server
       Releases	 more  recent  than 1.7.47 support the program scc-setup(1) to
       configure the Apache webserver.	Call this program without options  and
       it  will	 show its defaults.  Inspect these defaults and set the corre-
       sponding environment variables to adjust the  results  of  the  script.
       Refer to the manual page for more details.  Then call:

	  scc-setup -a

       to  activate the web-interface of scc-srv.  This adds the file scc.conf
       to the apache conf.d directory.	Its default contents are:

	  ScriptAliasMatch "^.*/([^/]*)/cgi-bin/scc-wrapper.cgi" "/var/opt/scc-srv/data/www/$1/cgi-bin/scc-wrapper.cgi"

	  Alias /scc /var/opt/scc-srv/data/www
	  <Directory "/var/opt/scc-srv/data/www">
	     Options -Indexes FollowSymLinks +ExecCGI MultiViews
	     AllowOverride AuthConfig
	     Order allow,deny
	     allow from all

       The apache webserver is restarted to activate  this  configuration  The
       ownership of the files and directories in the scc-srv data hierarchy is
       set to the user running the apache webserver.

       Note that there is no reason to convert an existing scc-srv to the  new

       To  restrict  the  access  to  the  SCC-data in directory /var/opt/scc-
       srv/data/www/All, you can use an .htaccess file in this directory.

       Access to the data is achieved by  a  cgi-script,  called  by  the  web
       server.	This script, scc-wrapper.cgi(1) is located in the cgi-bin sub-
       directory of /var/opt/scc-srv/data/www/All The images,  shown  in  some
       pages, are located in the subdirectory custom.

       For Apache 1.x, the ScriptAliasMatch has to be replaced with:

	  ScriptAlias prefix/All/cgi-bin/scc-wrapper.cgi \
       Where  prefix  has  to  be replaced by the path under the documentroot.
       This value is also supplied with the -d option of scc-summary(1).  This
       line has to be repeated for each realm you create.

       On  a  slitaz  2.0  system installing lighttpd and lighttpd-modules was
       sufficient to run scc-srv after using scc-setup(1).

       To save space, the SCC-data of a system in a realm consists of symbolic
       links  to  the realm All.  When you use only one realm ( All ), you can
       do without the option FollowSymLinks in the configuration-file  of  the
       web server.

       Note  that  the	data hierarchy of scc-srv is still present on a system
       after removing the software.  You have  to  remove  this	 hierarchy  in
       /var/opt/scc-srv manually.

       When  it	 is  not  desirable that all administrators have access to the
       SCC-data of all your systems, you can create additional sub-directories
       in  /var/opt/scc-srv/data/www  These sub-directories are called realms.
       Realms are maintained by means of scc-realm(1).	This program  supports
       the creation of a realm and adding/deleting systems to/from a realm.

       To  create  a  new realm Education and add the systems c012 and c014 to
       this realm, use the following command:

	  scc-realm -a -L c012,c014 -c Education

       To generate the summaries for this new realm, scc-update(1)  is	called
       after  creating the directory and the links for the SCC-data.  Refer to
       the manual page for more detailed description of setting up realms  and
       their permissions.

       It  is  possible	 to populate realms automatically by means of the file
       scc-auto-realm.conf in the custom subdirectory of a  realm.   Based  on
       hostname, OS and domain systems are (de)selected for a realm.  The sys-
       tems in the All realm are the base for this  process.   Refer  to  scc-
       update(1) for more details.

       You have to edit the file Education/.htaccess to restrict access to the
       Eduction realm.

       When you followed all the steps mentioned before in this document,  you
       should  be  able to see the start-page of SCC.  Use the path you speci-
       fied with the WWW_PATH environment variable determined by scc-setup(1).

       This start-page allows access to the realm All and to the documentation
       of scc-srv.  Note that new realms that are  created  with  scc-realm(1)
       are  only added automatically to this page as long as a specific marker
       is preserved in the file index.html.

       You are now ready to run a scc-client and transfer data to the  server.
       Choose  a  client-system	 and  a transfer-option and run scc.  Then run
       scc-update(1) on the server to transfer the data to the web-server  and
       to generate summaries.  When all the above steps have been followed, no
       options need to be specified.

       Use the -A option of scc-realm(1) to archive (part of) a realm.	 With-
       out  the	 -w  option, no html files are archived and only snapshots and
       logbooks are copied.  Combining these files with tar and compression is
       not part of the software.  When the -w option is used, a minimal webin-
       terface is provided by scc-summary(1) in the specified directory.  Pro-
       viding  and  limiting access to the archive directory is not handled in
       the software and left to the administrators.

       When system foo is no longer in use and you need to  archive  its  data
       for  compliancy,	 you  can  archive the data and then remove the system
       from the webinterface by means of one command:

	  scc-realm -A /path_to_archive -d -L foo All

       Replication and duplication have not been built into the software,  use
       rsync  to  achive  this.	 Another option is to transfer the data to two
       different scc-srv sites.	 When you are using scc-pull(1) to collect and
       transfer	 the  data,  use  the -k option on the first call to make sure
       that the clients preserve the data.  On the  second  run,  use  the  -n
       option  to  avoid  that	the  clients collect the data again.  When the
       clients push the data to the server, you have to schedule scc twice  on
       the clients and use the -n option for the second invocation.

       The  server  receives  SCC-data from clients by email, scp, ftp, rcp or
       cp.  A Denial Of Service is possible by frequently sending large	 snap-
       shots  of  fake	hosts  to  the	SCC  server.   Furthermore, anyone can
       (re)send SCC-data from any system to the SCC  server.   Therefore,  the
       SCC server should only be deployed in a trusted network.

       When the client sends its data using email via the internet, the sensi-
       tive data should be encrypted.  This is achieved by generating  a  pair
       of public/private keys on the server.  The clients encrypt the SCC-data
       using DES3 with random pass phrase.  The pass phrase is encrypted using
       RSA  with  the  public  key  of	the  server.  The server receives both
       encrypted files and uses its private key	 to  obtain  the  random  pass
       phrase  used  by	 the  client.	Finally,  this	pass phrase is used to
       decrypt the SCC-data.  To generate the keys on the server, use the fol-
       lowing commands:

	  # cd /var/opt/scc-srv
	  # umask 077
	  # mkdir encryption
	  # cd encryption
	  # openssl genrsa -out private_key 2048
	  Generating RSA private key, 2048 bit long modulus
	  e is 65537 (0x10001)
	  # openssl rsa -in private_key -out public_key -pubout
	  read RSA key
	  writing RSA key
	  # ls -l
	  total 16
	  -rw-------	1 root	   sys	     1679 Sep 29 08:51 private_key
	  -rw-------	1 root	   sys	      451 Sep 29 08:52 public_key

       The  file public_key should be transferred to the SCC-client.  The path
       of the file private_key should  be  added  to  the  file	 /var/opt/scc-
       srv/conf/scc.conf with the following keyword:


       After   transferring   the   public   key   to	the   client  in  file
       /var/opt/scc/encryption/public_key the data can be encrypted  and  sent
       with the command:

       /opt/scc/bin/scc			       \
	    -k /var/opt/scc/data/encryption/public_key	 \
	    -s scc-transfer@srv.dom

       When  the client cannot use a mail-client, use the -i and -p options to
       send the data via SMTP.	Refer to the  manual  page  of	scc  for  more

       First  of  all,	check the URL you are using to display the data.  This
       should correspond with the path you specified (once) with the -d option
       of scc-update(1).

       When  the  data	of  a  scc-client  does not show up, you can check the
       directories under /var/opt/scc-srv/data/transfer whether the  raw  data
       arrived.	  Also	check any possible errors reported by the clients when
       sending the data.  scc-transfer(1)  logs	 its  actions  when  the  file
       /var/opt/scc-srv/data/log/scc-transfer.log  exists.   Create  this file
       and try to send the data again.

       The log files of the web server can also be checked.  When  the	start-
       page  of	 SCC and the start-page of the realm All show up, but the cgi-
       script does  not	 function  correctly,  check  the  file	 /var/opt/scc-
       srv/data/log/scc.cgi.log.   scc.cgi(1)  logs  its  actions in this file
       when it exists and is writable by  the  user  running  the  web-server.
       Create this file and try to access the URL again.

       A  restrictive  selinux	can also avoid the display of SCC-data.	 Check
       the appropriate logfile to find out  whether  selinux  is  causing  the

       You can also use the program scc-debug(1) to obtain additional informa-
       tion concerning scc-srv.

       When this does not give any clues, you can  edit	 the  cgi-scripts  and
       trace  their  execution.	  Add  the  following  to  the	start  of scc-
       wrapper.cgi or scc.cgi:

	  exec 2>/tmp/scc-log
	  set -x

       On abnormal termination of a run of a scc-client, the snapshot will not
       be  complete and the next run will produce huge differences.  To remove
       these differences, you have to edit the log file on the client.

       The clients of SCC try to use gzip or compress to reduce the amount  of
       data  sent to the server.  This means that the server has to be able to
       access both gunzip and uncompress.  To install  uncompress  on  RedHat,
       install ncompress.

       The  SCC-data  that  is	sent by email, is uuencoded.  The server needs
       uudecode.  To install uudecode on RedHat, install shar-utils.


       The programs of scc-srv have the following relationship:

       - scc-receive-mail: move SCC-data to transfer-area
	 activated by /etc/aliases
       - scc-update: main update of data on website
	 activated by cron, this program calls:
	    - scc-transfer: move data from transfer-area to website
	    - for each realm:
		 - scc-rules: check rules
		 - scc-summary: update general summaries
		 - scc-smt: generate summary of System Tools
       - scc-pull: run scc on clients and pull files to server
	 this program calls:
	    - scc-update
       - scc-wrapper.cgi: wrapper for web-interface
	 activated by user, this program calls:
	    - scc.cgi: handle web-interface
		 - search/show summaries/log
		 - scc-syscmp: compare parts of snapshots
		      - scc-baseline: extract part of snapshot

       The HTML-files have the following relationship:

       - scc-help/scc*.html: help-files for the userinterface of scc-srv
	 part of distribution
       - scc-help/scc-srv/scc*.html: manual pages for scc-srv
	 part of distribution
       - scc-help/scc/scc*.html: manual pages for scc client
	 part of distribution
       - scc-help/scc-win/scc*.html: manual pages for scc-win
	 part of distribution
       - index.html: main-menu to realms
	 part of distribution, contains a specific line to assist scc-realm(1)
       - All: directory of main realm
	 part of distribution
	    - index.html: interface to snapshots/log files and summaries
	      calls scc-wrapper.cgi to perform action and display data
	      generated by scc-summary
		 - scc-summary-<cat>: summary of snapshots sorted on <cat>
		   supported categories are:
		      - host
		      - OS
		      - model
		      - last run/change
		   generated by scc-summary
		      - scc.<host>.html: snapshot of <host>
		      - scc.<host>.log.html: logbook of <host>
			transported by scc-transfer
		 - scc-log-index.html: summary of changes per day
		   generated by scc-summary
		      - scc-log-<date>.html: summary of systems with changes on <date>
			generated by scc-summary
			   - scc.<host>.log.html: logbook of <host>
			     transported by scc-transfer
		 - scc-rules-index.html: summary of messages
		   generated by scc-rules
		 - scc-smt-index.html: summary of System Tools
		   generated by scc-smt
		      - scc.<host>.smt.html: System Tools for <host>
			generated by scc-smt
		 - dynamic data with results of searching
		   generated by scc.cgi (called by scc-wrapper.cgi)
		      - scc.<host>.html: snapshot of <host>
		      - scc.<host>.log.html: logbook of <host>
			transported by scc-transfer
		 - dynamic data with results of comparing snapshots
		   generated by scc.cgi (called by scc-wrapper.cgi)


       A class-file contains lines with (parts of) classifications.  When they
       are not present, colons are added in front and at the end of each clas-
       sification.  Lines starting with "#" are treated as comments.

       To extract the boot configuration and start/stop links out a  snapshot,
       use the following class-file:


       Class-files  are	 used  by scc-baseline(1) and scc-syscmp(1). Check the
       snapshots to determine the classifications to be extracted.


       SCC is free software under the terms of the GNU General Public License.
       Copyright (C) 2001-2004 Open Challenge B.V., 2004-2005 OpenEyeT Profes-
       sional Services, 2005-2015 QNH.


       scc-baseline(1), scc-changes(1), scc-debug(1), scc-pack(1), scc-pull(1),
       scc-realm(1), scc-receive-mail(1), scc-rules(1), scc-setup(1), scc-smt(1),
       scc-summary(1), scc-syscmp(1), scc-transfer(1), scc-update(1),
       scc-wrapper.cgi(1), scc.cgi(1), scc-srv(5)


       $Revision: 5804 $