XCI-826 Testing Discussions

86 posts / 0 new
Last post
XCI-826 Testing Discussions
Delivery Effort Stage: 

This is feedback from the chapter 1 of the Installation Guide’ (XSEDE Globus Connect Server v5.4)’.

I’ve never done this, so I get them by just reading it at this point.

  • For the ‘Installation Guide’ (XSEDE Globus Connect Server v5.4) , there is “...using the globus-connect-server54 installation…” phrase in the very first section.  Is the “globus-connect-server54” correct?   (54 or 5.4 or none?)
  •  For the ‘Installation Guide’ (XSEDE Globus Connect Server v5.4) , about the 1.a, it seems suddenly appeared without an anchor point from the installation doc. It would be helpful to point it if applicable like 1.b did with “.. In Section “4.2.2. Setup the Endpoint ..”.
  •  For the ‘Installation Guide’ (XSEDE Globus Connect Server v5.4), about the 1.d and 1.e, I was wondering if this has an anchor point or is this after a certain step?   If applicable, it would be helpful to point to it or comment on it.

Hey Jonathan,

The name “globus-connect-server54” is not important, so I removed it.

I added links to anchor points as you suggest.


There is no direct anchor in the Globus installation doc for the custom certificate documentation--I have it as 1.a in the doc because it is relevant for some sites, though it could happen at any point in the installation.

With regard to 1.d and 1.e (actually now 1.e and 1.f, as I added a note as 1.d), they are to be completed after you finish section 4.7 of the Globus doc.  Section 1 of the XSEDE Globus Connect Server v5.4 Installation Guide covers through section 4.7 of the Globus GCS install doc--section 2 covers the material that starts in section 5 of the Globus doc.  I've added a note at the top of section 1 of the XSEDE doc that it covers through section 4.7 of the Globus doc.

Yes, that would help.

This is feedback from the chapter 2 of the ‘Installation Guide’ (XSEDE Globus Connect Server v5.4)

  • At the beginning of chapter 2, right after the dotted lines, there is a phrase “Refer to the Design for XSEDE SP Deployment of Globus Connect Server 5.4 for a more in-depth…” .  When I clicked the link, it led to the “ Primary access - For members of XSEDE-allocated projects“ section of another doc (Design for XSEDE SP Deployment of Globus Connect Server 5.4) .   For sometime, I got confused when I was scrolling down from the section.    It would be better to show contents from the 5.1.2  section.  As I am reading, I was wondering about the term “collection” and it turned out that it was under the 5.1.2 section of the other doc.   I also found the term “storage gateway” which is good to understand ahead. 
  • At the beginning of Chapter 2, right before 2.3.1 , there is a phrase “The configuration of a storage gateway and mapped collection for Science Gateways is very similar to that from Option 1 ... “.   It made me wonder what is the ‘Option 1’.  Does it mean option at 2.2 or the previous option (guest collection) ? Probably good to update the word.

I replaced the link to with 5.1.2.

We clarified the point right before 2.3.1. You are correct, it's similar to Guest Collections.

This is a question.

In the Globus Connect Server v5.4 - Integrated Testing Plan doc ( https://docs.google.com/document/d/1YaqxkS-AtIin0VYl5Pt_OkHW5LJJw1SLqbmg... ) , at the "Testing Setup" section, I see the "/local/xsede-*-storage/*" paths are listed. However, I don't have access permission to the "/local/" path.
Is this fine to proceed or do I need access permission to those paths?

I just fixed the permissions so that you have access to these two paths that you do need:

[jkm@xcidev1 ~]$ ls -l /local/xsede-projects-storage/
total 0
drwxr-x---. 2 jkm users 29 Apr  7 13:37 jkm
[jkm@xcidev1 ~]$ ls -l /local/xsede-guestdata-storage/
total 0
drwxr-s---. 2 jkm guestdata 29 Apr  7 14:40 jkm

In the 'https://docs.globus.org/globus-connect-server/v5/#install_section' , at 4.2.1-6 , I got stuck because I don't know what to enter for the 'Client Secrets' Description field.   I think it would be helpful to tell a user what to enter for it in our 'XSEDE Globus Connect Server v5.4 Installation Guide' doc in the '1. Creating an Endpoint' section after a.  
What should I enter for it?  Where do I get it?

I tried empty or a string, I get an error message "Failed to get client secret. Please try again."


explains how to obtain the client secret in section 4.2.1. When you follow those instructions the developers.globus.org application displays a "Generate New Client Secret" button that you press.


Yeah, I followed the steps and getting the error at #6 step at 4.2.1 section on the doc. 

Since I can’t attach pictures, I just sent the related email to you.

I logged out and logged back to globus.org and tried again, and it worked this time!

It must have been a system hiccup.  I did exactly the same and I get the Client Secret ID this time.  No idea.

Anyway, if you care about a new user's confusion, I think it's still helpful to mention that the 'Description' can be any string.  If I knew that, I probably would have tried again without getting stuck on such a strange hiccup.  


This is just to record my procedure here. ( only the steps that are not totally clear to me if it's a step must do or an optional step. )  

  • NEWLY ADDED  -----------------
  •  The instruction https://software.xsede.org/development/xsede-oauth-mapfile/latest/INSTALL ( from 1.e step of the installation guide ) is a bit confusing.  I had to read multiple times to make more sense.  I hope it makes better sense for the admins.   Here is my confusion list:
    •  From "### RPM Install" step , I see that 'xsede-oauth-mapfile' rpm got installed under '/usr/local/share/utils/xsede_user_mapfile/' as stated. However it also says the default location for the mapfile is '/usr/local/etc/grid-security/xsede-oauth-mapfile' .  Where should it be?
    • From '### Configure xsede-oauth-mapfile' step,  I found the 'etc/xsede-oauth-mapfile-config-template.json' file from the 'xsede-oauth-mapfile' rpm installed in the previous step.  I got confused with the path for a while.   It would be helpful to mention that the template .json file is from the 'xsede-oauth-mapfile' rpm installed in the previous step. 
    • Question, the template json file looks as below.  Do I use the RESOURCE "bridges.psc.xsede" or other?
      •      "XA-AGENT": "spacct",
            "XA-RESOURCE": "bridges.psc.xsede",
            "XA-API-KEY": "012345abcde012345abcde"
      • As I read back to the "

        XSEDE Globus Connect Server v5.4 Integrated Testing Plan" doc , at step Test1.2 , I see "On xcidev1 pretend to be expanse.sdsc.xsede.org". Does this mean set the "XA-RESOURCE" to 'expanse.sdsc.xsede.org' ?  

    •  At step "## Request API Access Key" , it would be helpful to rephrase the explanation to " obtain one by clicking the "Generate APIKEY" link and follow the instructions at https://xsede-xdcdb-api.xsede.org/.   I had to read it and click links for a while.
    • Where should I locate the 'etc/xsede-oauth-mapfile-config.json' file?    The same path  as the template json file from the rpm installed?  I am asking it because the path ' /usr/local/etc/grid-security/xsede-oauth-mapfile' is also mentionoed in the 'RPM Installl' step and get confused. 



  • At the "4.5. Log into the endpoint" , it looks like an optional step.  I ran the command, went to the link and enter the "Native App Authorization Code" from the web link.  The command displayed "You have successfully logged into GCS endpoint ...".  Not sure what to do after that, so record it here.


  • At the "4.7. Set the endpoint as managed", I ran the command but got an error.   Thus, I requested adding the endpoint ID to the XSEDE Globus subscription via help@xsede.org according to step '1.d' in "XSEDE Globus Connect Server v5.4 Installation Guide".  (this is just testing so a little confused )


Hey Jonathan,

4.3 starts the initial server and transfer node while 4.4 adds additional data transfer nodes. One endpoint can have multiple transfer nodes. 

If you did 4.3 and it worked, everything is good.

We clarified Step d. regarding 4.7. Please let us know if the instructions are still not clear.

Okay. Got that about 4.3/4.4.   So it sounds like I don't need to run it again for this test purpose. 

For Step d. regarding 4.7, yes it sounds clear.

This is the reply to comment #12 (not #14 as this header shows).

I updated the above comment.  The "NEWLY ADDED  -----------------" section is added for your reply. 

( Not sure if you get a notification when I 'Edit' the content )


Please avoid updating comments, except perhaps to make spelling corrections or minor clarifications. New feedback should get a new posting.

I'll reply to Comment #12 directly.



  • Corrected and clarified the location of the generated /etc/grid-security/xsede-oauth-mapfile
  • Added instructions in the Testing Plan for setting the XA-RESOURCE value
  • Added clarifications on the Request API Access key instructions



For #2, I can see the update in the Testing Plan doc.  Looks good.  I will use 'expanse.sdsc.xsede.org '.

For #1 and #3, I can't find them.  Where are those updated?  I don't see any change in the https://software.xsede.org/development/xsede-oauth-mapfile/latest/INSTALL 

#1 and #3 will be updated at the link you mention sometime before testing can finishes.

OK. I'll continue.

In an offline e-mail discussion with Michael we were reminded that the XDCDB Resource Name and XA-RESOURCE used with the API can ONLY have one API key. It should be either the offiical XDCDB Resource Name for production resources, or any hostname used for testing or to access the API from. This value is not linked to the resource name that mapping are accessed for. For example, the XDCDB Resource Name and XA-RESOURCE for testing can be xcidev1.dyn.xsede.org where testing is happening, so that the API key provided is for that resource, but the resource for which mappings are retrieved during testing can be expanse.sdsc.xsede.org or any other resource.

This clarification has been added to xsede-oauth-mapfile 1.0.5-6.

About step b at https://software.xsede.org/development/repo/repoconfig.txt.  (from the 1.e Installation Guide),  when I run the "rpm -i ..." , I get an error as below.  

$ rpm -i http://software.xsede.org/development/repo/repos/XSEDE-Development-confi...
error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Permission denied) 


I thought to try 'sudo' but I am following the instruction.

Let me know how to handle this.

Software install  using rpm can only be done as root. Try prefixing the command with "sudo <rpm_command>" to run as root.

You have been authorized to run as root using sudo on xcidev1.

Okay. It worked.   Thanks.

( Btw, I don't get an email notification when you reply here. )

Click on the "Subscribe" link at the top of this page: https://software.xsede.org/node/3895#comment-532

This is at step 2.1.1  Creating a Storage Gateway' from the GCS Install Guide.

I am getting errors for creating the Storage Gateway.

What I did:

1. I created a cmd fille 'StorageGatewayCreate.sh' as below:

  • globus-connect-server storage-gateway create \
        posix "XSEDE Expanse Primary Storage Gateway" \
        --domain xsede.org \
        --authentication-timeout-mins $((60 * 24 * 11)) \
        --identity-mapping file:/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json \
        --restrict-paths file:/home/jkm/TestCmds/InstallGuide_2/StorageGatewayCreate/path_restrictions.json

2. Run it without sudo

  • $ ./StorageGatewayCreate.sh

ERROR1:  PermissionError: [Errno 13] Permission denied: '/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json'


3.  Run it with sudo

  • sudo ./StorageGatewayCreate.sh

ERROR2: Error: You must login first, before running this command!

4. Login

  • $ globus-connect-server login localhost
    You are already logged into a GCS endpoint.


5. Run it again with sudo

  • sudo ./StorageGatewayCreate.sh

ERROR: Error: You must login first, before running this command!

I am still getting error even I am already logged in.




FYI, it created the storage gateway when I removed the "--identity-mapping file:...." line from the above creating command for a testing purpose,   I ran it without 'sudo'.


The --identity-mapping argument is required and the storage gateway will not work without it. Please delete the storage gateway you created without using --identity-mapping.

Then, please run all the globus-connect-server commands as root using sudo.

So this includes logging in to the GCS, right?


The 2.1.1 Creating a Storage Gateway finally worked!

Btw, this is a comment from my experience related to this issue.

I think it would be very helpful to mention that something like "Login to the GCS before running any Globus commands." in the installation doc.

I didn't know that concept before this step.  I happened to log in to the GCS when I was following the https://docs.globus.org/globus-connect-server/v5/#install_section doc , step 4.5.  However, it looks like an optional step which I almost bypass at that time.

The last sentence in JP's reply above is incorrect.  These commands should NOT be run as root or sudo.  The globus-connect-server login command, and all subsequent commands, are intended to be run in userspace, not as root.  (The globus-connect-server node setup command does need to be run as root or sudo, though.)


The last sentence in JP's reply above is incorrect.  These commands should NOT be run as root or sudo.  The globus-connect-server login command, and all subsequent commands, are intended to be run in userspace, not as root.  (The globus-connect-server node setup command does need to be run as root or sudo, though.)

I guess, the error “you must login’ occurs when using ‘sudo’ with the above cmd because I logged in GCS without ‘sudo’ which probably means the login id is ‘jkm’.  

However, ‘sudo’ will change id as root id, so it may not match.

Should I login to GCS with the ‘sudo’ command as root id?

(ex: $ sudo globus-connect-server login localhost )


Before I try it, I am asking because the root access is too powerful and may mess up something if it’s not meant to be used.


It is easier to just run "sudo -i -u root" to create a root shell, and then run all the commands from that shell, instead of running sudo for each command.

I think it would be surely helpful to mention it in the Test Plan doc or Installation Guide where it fits.  Ex: "run all the commands (linux and globus) in the root shell ( sudo -i -u root) or as a root." from the beginning of the doc. 


We've clarified in the instructions which steps much be run as root, and which should not. If the instructions are still not clear please let us know.

I am done with 2.1 and I am at step 2.2  creating Guest Collections from the GCS Install Guide.


For the command to create a storage gateway for the Guest, what should I set for “<sharing group>”?   

This question also is for the Science Gateways.



Also, can I use the same “path-restrictions.json” file I used for creating the Primary section for the Guestt?   

This question also is for the Science Gateways.


Btw, I really have to work on the other project at NCSA at least today and tomorrow, so I will focus back on this after that. (I told Shava about it)

It seems that this Test1 is the most important and takes the longest time to go through. I think the most critical part is done so I probably can finish T1 by Monday and all the rest by Tuesday. Let me know if I guessed wrong about the T2,T3,T4 time line.


Hi JP,   I will need answers for these questions to continue for 2.2 and 2.3 steps.  After that, I can wrap up with T1. 


For Q1: I just updated the Testing Plan to explain to use the "--user-allow <testing_username>" option instead of the "--posix-group-allow".

For Q2: Notice that there are different path restrictions for Primary, Guest Collections, and Gateways, so you will need different path restriction files

I saw Lee’s comment about not using ‘sudo’ from a certain point, and I see that step 1.d is added for that in ‘XSEDE Globus Connect Server v5.4 Installation Guide’ doc.

Does this mean I should have not used ‘sudo’ for “yum install xsede-oauth-mapfile” as well at the  https://software.xsede.org/development/xsede-oauth-mapfile/latest/INSTALL page?


I guess not. 

I removed it and try to install it again without 'sudo' and got "Error: This command has to be run with superuser privileges (under the root user on most systems)."

I will use 'sudo' again for “yum install xsede-oauth-mapfile” for the above step.

No. You have to be root to install software. The XSEDE Installation Guide step 1.a. now details what steps require root and which should not use root.


After reading comments about not using 'sudo', I went through again from the step 1.d of "XSEDE Globus Connect Server v5.4 Installation Guide". 

It worked this time for creating the Storage gateway(2.1.1) and Collection (2.1.2) for Primary access as shown below:

( notice that I didn't need the sudo. )



$ globus-connect-server whoami
Username      | Name         | ID                                   | Email
------------- | ------------ | ------------------------------------ | ----------------
jkm@xsede.org | Jonghoon Kim | ecb182f4-96b1-4cd1-83b5-d6ca9255d658 | jkm@illinois.edu



$ globus-connect-server storage-gateway list
Display Name                        | ID                                   | Connector | High Assurance
----------------------------------- | ------------------------------------ | --------- | --------------
Expanse XSEDE Useer Storage Gateway | 4806fe58-7f80-4ae2-8ba6-169edee49c3e | POSIX     | False



$ globus-connect-server collection list
ID                                   | Display Name                       | Owner         | Collection Ty
pe | Storage Gateway ID
------------------------------------ | ---------------------------------- | ------------- | -------------
-- | ------------------------------------
d0710bd9-ad1c-4da7-bf65-0f59f020ad65 | Expanse XSEDE User Data Collection | jkm@xsede.org | mapped
   | 4806fe58-7f80-4ae2-8ba6-169edee49c3e$ globus-connect-server collection list
ID                                   | Display Name                       | Owner         | Collection Ty
pe | Storage Gateway ID
------------------------------------ | ---------------------------------- | ------------- | -------------
-- | ------------------------------------
d0710bd9-ad1c-4da7-bf65-0f59f020ad65 | Expanse XSEDE User Data Collection | jkm@xsede.org | mapped
   | 4806fe58-7f80-4ae2-8ba6-169edee49c3e



At step “### Setup to generate using cron” of https://software.xsede.org/development/xsede-oauth-mapfile/latest/INSTALL ,  it would be helpful to add a comment about what the user can expect as the result of running the command.    For example,  “If the command succeeds, the result file will be generated as “/etc/grid-security/xsede-oauth-mapfile” and it will contain peoples’ email id lines. This also means it succeeds to access XDCDB API.”   This way if it didn’t work the user will know what to do next. I was wondering if I am allowed to run it manually for a test before setting it for cron.

At step 2.2 of "XSEDE Globus Connect Server v5.4 Installation Guide" ,  I got an error by running the creating Storage-gateway for the Guest collection shown below.


globus-connect-server storage-gateway create \
   posix "Expanse XSEDE Guest Collections Storage Gateway" \
   --domain xsede.org \
   --identity-mapping \ file:/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json \
   --restrict-paths file:/home/jkm/TestCmds/InstallGuide_2/2_Guest/path_restrictions.json \
   --allow-guest-collections \
   --user-allow jkm


Usage: globus-connect-server storage-gateway create posix [OPTIONS]

Error: no such option: --allow-guest-collections

The spelling looks correct for the "--allow-guest-collections". 

Do you see any issue with it?


  "DATA_TYPE": "path_restrictions#1.0.0",
  "read_write": [
  "none": [


Ah, yes, I put that option on the wrong command. It's been removed from this storage-gateway command and put on the collection command.


Thanks for the update.

So I removed ' --allow-guest-collections' from your update.

Now I have a different error.


Error: the string ' file:/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json' is not valid JSON

This is the same json file path I successfully used for creating Primary storage-gateway&collection at step 2.1.


$ cat /usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json
  "DATA_TYPE": "external_identity_mapping#1.0.0",
  "command": [



This error was from my mistake. 

After fixing this from 

"--identity-mapping \ file:/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json \"


"--identity-mapping  file:/usr/local/share/utils/xsede_oauth_mapfile/etc/gcs-mapfile-lookup.json \"

It works.  I'll continue.


Log in to post comments