[XCI-331] Give option in IPF to pre-populate the position files for activity scanning Created: 02/14/2018 Updated: 07/12/2018
|Project:||XSEDE Cyberinfrastructure Integration|
|Public activity link:||https://software.xsede.org/display/xci-331|
|Track status in meeting:||no|
When IPF reads activity logfiles to parse job info changes, it scans the logfile directory for all "new" files, i.e. ones it hasn't already processed. The issue is that some of these directories hold logs that go back years (~1200 files) and IPF can't process them all before the AMQP connection times out. The solution is to allow the SP to seed a position file that marks all (or most) of the extant files as already processed, and go from there.