UREP Prioritization Rating FAQ

Guidance on rating proposed Use Cases

 

Can I edit my ballot after voting?

Yes.  You can go back to the voting page at any time until the voting closes and change or complete your ballot.

Can  you explain the rating scale?

Vote ’N/A’ if you have no information on which to base a rating.

If you have information on which to base a rating:

  • Vote 1 if you believe there is no value for XSEDE providing this capability (needs are already satisfied, audience is too small, clearly negative ROI, etc.)
  • Vote 2 if you believe there is little value for XSEDE providing this capability (yeah, someone will probably benefit for but it’s unclear whether ROI is positive)
  • Vote 3 - 5 if you believe there is sufficient ROI to do it; differentiate by low, moderate, or high impact/ROI

 

Are you looking for approval only, or discrimination: do we value this over that?

We're looking for a sense of the importance for each independent item. Please assign the best match (N/A or 1 thru 5 stars) to each item independently.

How should I rate a case where the current plan is only to do an assessment of the possibilities?

Please rate these cases according to whether or not you think the proposed capability itself is useful and important. 

For some Use Cases, the first step is to determine what the options are for implementing it.  In those cases, the "Capability Delivery Plan" (CDP) reflects that, with language like "At this time, the delivery plan is to search for, identify, and evaluate available technologies that XSEDE could use for this...".   

This doesn't mean, of course, that we are just going to do an assessment of the options and then mark that Use Case completed.  The results of the assessment will inform the way we proceed with bringing that Use Case into production.

I want to provide more info about my vote for a Use Case. How can I do that?

Use the comments field to add any comments or clarifications you'd like to your vote on any case.

Do I have finite resources to commit (I know the answer is yes in reality, but how would you like me to think about it?)

Only factor the amount of effort for the individual item. Reality is too complicated to allow factoring in our staffing level, for many reasons:

  • We're also required to do work that isn't prioritized by the UREP but is required by the NSF, our proposal, or operational realities.
  • We're also responsible for maintenance effort, which is hard to predict.
  • We don’t always have to do the work ourselves! (We have partners who often help us out with things that are also in their interest.)
  • We’re not 100% sure when our already-underway work will be finished.

How many person-months are available to commit to these items? Taking that into account would force me to allocate with discrimination, if that was your goal.

Don't worry about what resources are available. We're asking for your thoughts on how beneficial each feature/idea would be to the community. NSF wants us to consider the cost/benefit of our work, and you're an important source of the "benefit" part of that.

In practice, whenever we have resources available to work on something new, we look at the UREP priorities to help us decide what to do next. We'll do as much as possible in higher to lower priority order, based on the resources on hand. We'll avoid things the UREP rates as "1" (don’t do), and we'll think very carefully about the "2"s.