Skip to content Skip to navigation

JIRA Issue

[#XCI-331] Give option in IPF to pre-populate the position files for activity scanning

[XCI-331] Give option in IPF to pre-populate the position files for activity scanning Created: 02/14/2018  Updated: 07/12/2018

Status: Proposed
Project: XSEDE Cyberinfrastructure Integration
Component/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Eric Blau Assignee: Unassigned
Resolution: Unresolved Votes: 0

XSEDE Priority: -
Public activity link: https://software.xsede.org/display/xci-331
Track status in meeting: no

 Description   

When IPF reads activity logfiles to parse job info changes, it scans the logfile directory for all "new" files, i.e. ones it hasn't already processed.  The issue is that some of these directories hold logs that go back years (~1200 files) and IPF can't process them all before the AMQP connection times out.  The solution is to allow the SP to seed a position file that marks all (or most) of the extant files as already processed, and go from there.