Post DSR feedback here
Delivery Effort Stage:
Research Software Portal
Provided by ACCESS
GCSv5 Custom Certificates (for XSEDE) Design and Security Review feedback
Tue, 2019-12-17 12:54
#1
GCSv5 Custom Certificates (for XSEDE) Design and Security Review feedback
© ACCESS All Rights Reserved.
ACCESS is an advanced computing and data resource supported by the
National Science Foundation
and made possible through these lead institutions and their partners:
Carnegie Mellon University,
University of Colorado Boulder,
University of Illinois at Urbana-Champaign, and
State University of New York at Buffalo .
Administrative Login
Due to security reasons, I suspect we will want to scope SSL wildcard certificates to a specific subdomain, to limit exposure if one of the certs is compromised. The way DNS works, I would need to delegate additional subdomains to each XSEDE institution.
Example:
globus.psc.xsede.org
From there, institutions can create additional host/wildcard records and corresponding SSL certificates.
dtn1.globus.psc.xsede.org
*.globus.psc.xsede.org
For Option 1: Bullet 1, sub bullet 2: "Obtain a certificate and key for the research.example.edu and *.research.example.edu DNS names and register it with the GCS Manager API" Does that mean you would need to upload your certificate and key into the Globus "system"? I suspect that will break some policies somewhere.
The following answer to your question was provided by the Globus development team. (My own emphasis is added.)
"The certificate and key would be registered with the GCS Manager API running on the Endpoint run by the institution. It is used there by a web server to serve GCS Manager API and GridFTP data traffic. The certificate and key would be stored in an encrypted form in the appsync dynamodb for distribution between nodes. That data is encrypted with a key that Globus does not have any access to---each GCS Manager node has its own locally-generated key that does not leave the node."
The development team's answer above alludes to another new feature of Globus Connect Server version 5: the GCS Manager API (service), which is a new component in version 5. The GCS Manager API runs on the GCS host system and it serves a couple of purposes: (1) It assists the administrator in synchronizing the GCS configuration among all of the endpoint's nodes (when the endpoint has multiple data movers), (2) It enables the administrator to retrieve a copy of their GCS configuration data from Globus in the event that the local copy is lost. (Obviously, the admin needs to retain a copy of the local encryption key for this to be successful.)
> Are there any security or design concerns with XSEDE service providers "SPs" choosing the default GCSv5 DNS and certificate configuration?
[1a] The review document suggests that the deployment's endpoints trust the GCS Manager for configuration. The document also suggests that the certificate used to identify the GCS Manager is the same one used to identify the deployment's endpoints to clients. If this is the case, then a deployment's endpoints also extend their trust boundary to the data.globus.org DNS servers and Letsencrypt. Put another way, an adversary able to alter the data.globus.org DNS zone would be able to push arbitrary configuration onto all GCSv5 deployments. Maybe I'm mis-understanding the description of this component.
I'd feel better about keeping the PKI for management separate from server-to-user-client authentication; separate meaning each deployment has its own CA with a unique key (or self-signed end-entity cert since presumably the manager doesn't need to authenticate the clients), which is the only CA/cert that the endpoints will accept.
[1b] Letsencrypt also has restrictions on the rate that they will issue certs for a given domain. Unless prior arrangements have been made to bypass this limit, a situation may arise where some deployments can't renew their certificates because too many other deployments are doing the same thing. Also, keep in mind that getting a wildcard Letsencrypt cert also requires dynamic-dns, as the http challenge doesn't work for wildcards.
[1c] Does the user ever see the <random>.data.globus.org hostname? If so, would they ever care that they're connecting to a specific deployment, as opposed to some other one that appears to have the same files? This arrangement can mask from the user a situation that they're connected to the wrong deployment. If the user doesn't see that information anyway, it may not be as big of a concern.
[1d] Does the user ever have to pass a password to an endpoint or deployment? If yes, then the practice of providing a password to a resource using a hostname of <random>.data.globus.org will likely violate institutional security policy as well as creating a situation indistinguishable from a phishing attack
> Do SPs need a custom certification configuration option?
[2a] Assuming the above concerns are addressed, and the only purpose for the default cert option is for authenticating the HTTPS transport to users' browsers, we (SDSC) can work with Letsencrypt. Otherwise....
> If yes:
>Which one is preferred and satisfies the most requirements?
[3a] Option 2; Option 1 has a major drawback: it effectively puts an institution's sub-domain in the control of a 3rd party, which at the least, allows the 3rd party to issue DV certs for that sub-domain with no input or control from the institution. A follow-on consequence is that the sub-domain cannot be re-used for other purposes besides GCS until it's reasonable to assume that all DV certs that may have been issued have expired and thus cannot be used for MITM attacks on the new service(s).
> Are there security concerns or risks that should be mitigated for the preferred option?
> Are there design improvements you would recommend for the the preferred option?
[4a] As mentioned in 1a, we'd prefer to avoid trusting just any CA for providing deployment configuration. The CA that is trusted would be one that we control so that the chance of having a rogue cert used to change configuration is minimized if not eliminated.
[4b] Alternatively, allow for another method for synchronizing endpoint configuration. We're really good at pushing text files around securely in the environment that the deployment would run in.
The following answers were provided by the Globus development team.
[1a] The review document suggests that the deployment's endpoints trust the GCS Manager for configuration. The document also suggests that the certificate used to identify the GCS Manager is the same one used to identify the deployment's endpoints to clients. If this is the case, then a deployment's endpoints also extend their trust boundary to the data.globus.org DNS servers and Letsencrypt. Put another way, an adversary able to alter the data.globus.org DNS zone would be able to push arbitrary configuration onto all GCSv5 deployments. Maybe I'm mis-understanding the description of this component.
This was a maybe a little unclear in the document.
The endpoints nodes host the GCS Manager Service. There is a second service hosted by globus which the GCS manager interacts with to synchronize data between nodes in real time. The data which is stored in globus is protected in two ways: all data is encrypted by the nodes before sending the data to globus, and the data may only be queried with an OAuth token using the endpoint node's credentials.
The gridftp process interacts with the local GCS Manager service to get configuration information, but will only ever connect to the service running on the localhost (it explicitly does not use DNS). The data that it receives is always encrypted at rest.
I'd feel better about keeping the PKI for management separate from server-to-user-client authentication; separate meaning each deployment has its own CA with a unique key (or self-signed end-entity cert since presumably the manager doesn't need to authenticate the clients), which is the only CA/cert that the endpoints will accept.
[1b] Letsencrypt also has restrictions on the rate that they will issue certs for a given domain. Unless prior arrangements have been made to bypass this limit, a situation may arise where some deployments can't renew their certificates because too many other deployments are doing the same thing. Also, keep in mind that getting a wildcard Letsencrypt cert also requires dynamic-dns, as the http challenge doesn't work for wildcards.
Globus uses the DNS challenge with letsencrypt and wildcard certificates. Renewals aren't an issue: Per https://letsencrypt.org/docs/r
"Renewals are treated specially: they don’t count against your Certificates per Registered Domain limit, but they are subject to a Duplicate Certificate limit of 5 per week. Note: renewals used to count against your Certificate per Registered Domain limit until March 2019, but they don’t anymore."
[1c] Does the user ever see the <random>.data.globus.org hostname? If so, would they ever care that they're connecting to a specific deployment, as opposed to some other one that appears to have the same files? This arrangement can mask from the user a situation that they're connected to the wrong deployment. If the user doesn't see that information anyway, it may not be as big of a concern.
The hostname is published in the collection definition on globus.org, so it is visible via the Globus Transfer API and on the globus.org web pages which contain information about the endpoint. When using the globus.org webapp, there are options to interact with data from GridFTP servers in the browser, and those will use the endpoint domain.
[1d] Does the user ever have to pass a password to an endpoint or deployment? If yes, then the practice of providing a password to a resource using a hostname of <random>.data.globus.org will likely violate institutional security policy as well as creating a situation indistinguishable from a phishing attack
For GCSv5, all interactions are authorized using Globus OAuth which is an OAuth 2 implementation, so credentials are handled via instituational login applications and a token is generated to interact with the service (with a chance for user consent). There will be an optional add-on to provide an OIDC implementation for organizations which do not have an OAuth service that can be used with Globus Auth, but I don't think this would apply here.
On behalf of the PSC Domain and Certificate admins, we are amenable to "Option 2".
Shane
I find the scope of this document to be too narrow to fully understand the implications of the design. Of the two options presented, I feel that the second option is more reasonable for XSEDE SPs.