Auditing client usage, server logging, and the analysis scripts all look great.
We should also collect all the relevant logs in the central xci-metrics.xsede.org. I propose these additions to this design:
- Daily log rotation of all logs used in auditing (or weekly if you feel that is more appropriate)
- Daily sync'ing/scp'ing of all rotated logs to the xci-metrics.xsede.org server
- Duplicate installation of the analysis tools on xci-metrics.xsede.org. If you prefer, the design can just stipulate that all the scripts will be maintained in XSEDE's svn repo, and deploying a duplicate copy on xci-metrics.xsede.org can be part of the xsede-metrics implementation design rather than the design of this activity.
This will give us central long term repository for metrics data from all our services. We'll also be able to use those analysis tools on usage logs from other servers.