Validating that submitted data is semantically relevant is extremely important when trying to use or report on the data. Section 5.1.4 (or a section in the Implementation/Technical details) should further elaborate on how this validation will be performed. For example, will the SQLite database described in 6.3.3 contain a list of valid attributes and accepted formats? I assume that there will there be API calls to display this information? Since this data will not be stored in the XDCDB there needs to be a way to verify that the data that this process stores in the job attributes tables is valid and that another process did not store invalid gateway data using the same attribute names.
Section 5.1.3 states "The design SHOULD allow for the submission of free-form attributes not discussed in this document. The attribute name MUST be present in a white-list consulted during the submission process to avoid polluting the staging table with meaningless information." Where will the whitelist be stored? Doesn't having a whitelist preclude free-form attributes or does "free-form" refer only to the values and not the attribute names themselves?
My understanding is that the free-form attributes are limited to those that are listed in the design document, extra attributes if any will be thrown away.
That sounds reasonable as long as we include the list of supported attributes in the document and there is an API call to list them along with any formatting requirements.
That's not defined in the doc because we were not sure of how many extra attributes were going to get added. If it's a small number that doesn't change, a logical place to put the whitelist is in an array defined in the webapp's code.
Free-form refers to the values, not the attribute names. Sorry for the confusion.