JIRA Issue

[#XCI-567] Ingest Globus Transfer service usage data

[XCI-567] Ingest Globus Transfer service usage data Created: 04/04/2019  Updated: 10/28/2021

Status: Closed
Project: XSEDE Cyberinfrastructure Integration
Component/s: XSEDE Usage Collection Service
Fix Version/s: PY8 (Sep '18 - Aug '19), PY9 (Sep '19 - Aug '20)

Type: XCI Enhanced Capability Priority: Normal
Reporter: Lee Liming Assignee: Lee Liming
Resolution: Unresolved Votes: 0

Issue Links:
duplicates XCI-407 Track Globus Transfer usage using XCI... Cancelled
Target Operator:
XSEDE Community Infrastructure
XSEDE Priority: -
XSEDE Areas:
RACD Management
Public activity link: https://software.xsede.org/display/xci-567
Devel Repository:
Use Cases:
CAN-02: Managed data transfer, CAN-05: Record and summarize the use of a service
Effort and Costs:
Staff Name (Lastname, Firstname) Effort (person weeks) Roles or Contributions Status
Liming, Lee 0.2 tracking & reporting on work begun
Liming, Lee 0.2 one day to upload backlog of data since August 2011 complete
Liming, Lee 0.4 two days to develop a translation script from native record format to XSEDE standard record format none
Liming, Lee 0.6 three days to develop an automation mechanism for monthly ingests of new data from Globus none
Due by Activity Deliverable
Deployment Usage data in xci-metrics.xsede.org repository
Deployment Translator script from native record format to XSEDE standard record format
Deployment Automation mechanism for monthly ingests of new data
  • Click on "Deliverables" tab for URL.
Planned Launch Date:
Actual Launch Date:
Planned Design Review Date:
Planned Test Readiness Review Date:
Planned Complete Date:
Actual Complete Date:
Activity Lead: Lee Liming
Lead Tester: Shava Smallen


The Globus Transfer service is a discrete service that manages data transfers between Globus endpoints/collections on behalf of users. Users request transfers via the Globus webapp, CLI, or API, and the transfer service executes the transfers and manages QoS for the transfers. Users receive email notification when their transfers complete.

Usage data for the Transfer service is available from August 2011 to the present, and new data becomes available on a monthly basis. We should:

  1. Ingest the existing data into the XSEDE usage data service (XCI-187).
  2. Create a translator script from native record format to XSEDE standard record format.
  3. Set up an ongoing process for new usage data to be ingested.


There are comments for XCI-567 that can be viewed with XSEDE authentication.