Source Control
Source control provides a logical way to organize and control collaboration, revision, and release of source code. As a best practice, private or sensitive information (including but not limited to: personally identifiable information (PII), absolute file system paths, internal server host names or IP addresses, and usernames/passwords) should always be extracted from the source code and maintained separately.
Note: Git maintains a repository history and any commits containing sensitive information must be purged from the repository history. Refer to the Git Help on Rewriting History.
Where should I put my source code?
The USGS only supports Git for source code control. There are three unique hosting platforms for Git available to USGS employees. For more information contact GS Help Git (gs_help_git@usgs.gov).
Each platform component provides similar but distinct functionality. When deciding which component to use, please see the following information:
Sources/Usage: Public Domain. View Media Details
USGS GitHub: USGS Organization on GitHub [EXTERNAL]
This component is a commercial off-the-shelf hosting solution provided and hosted by GitHub. GitHub supports integrated issue tracking, project boards, and third-party integrations among other features. This component is publicly accessible. Private repositories are not available in this component. Any person – regardless of USGS affiliation – can create a GitHub account and collaborate on USGS projects by submitting pull requests.
This component is a commercial resource governed by its terms of service as amended for government use.
USGS OpenSource GitLab
This component is a commercial off-the-shelf installation of GitLab Community Edition (CE) provided and hosted in the cloud by the USGS Cloud Hosting Solutions (CHS) team. This component supports integrated issue tracking and project boards among other features. Some listed features may not be enabled in the “OpenSource GitLab” component.
This component is publicly accessible from anywhere on the internet. Projects identified as “Public” within this component are viewable by all users regardless of authentication. Projects identified as “Internal” within this component are viewable by any authenticated user. Projects identified as “Private” are viewable only by authenticated and authorized users. Default project visibility is “Internal”.
All USGS personnel automatically have accounts on this system. Simply log in with your Active Directory credentials or smart card. External collaborator accounts may be requested by a USGS sponsor and require the collaborator use two-factor authentication through a free third-party application (or open source alternative). Authorization is managed through project, user, and group permissions assigned by system and project maintainers.
This component is hosted in the CHS Amazon Web Services (AWS) Cloud and leverages AWS managed services to provide the highest available up-time and recovery. Data stored on this component are backed up nightly. System and application patches and updates are applied monthly.
Note: USGS Scientific Software Policy - All authoritative copies of Scientific Software intended for approved release must reside here.
USGS InnerSource GitLab
This component is a commercial off-the-shelf installation of GitLab CE provided by GitLab and hosted in the cloud by the USGS CHS team. This component supports integrated issue tracking, project boards, container image registry, and shared task runners among other features. Some listed features may not be enabled in the “InnerSource GitLab” component.
This component is only accessible while on the DOI network or using the Pulse VPN. Projects identified as “Public” within this component are viewable by all DOI network users regardless of authentication. Projects identified as “Internal” within this component are viewable by any authenticated user. Projects identified as “Private” are viewable only by authenticated and authorized users. Default project visibility is “Private”
All USGS personnel automatically have accounts on this system. Simply log in with your Active Directory credentials or smart card. Authorization is managed through project, user, and group permissions assigned by system and project maintainers. External collaborator accounts are not supported.
This component is hosted in the AWS Cloud and leverages AWS managed services to provide the highest available up-time and recovery. Data stored on this component are backed up nightly. System and application patches and updates are applied monthly.
Lifecycle Stages of Source Code
Code writing is often an iterative process and your software source code will likely fall into all of the categories below at some point in the process. Here are some helpful tips about Git and lifecycle stages of software.
Pre-release Development
This source code can exist in folders on local workstations or user account namespace on any USGS GitLab project.
- Private
- USGS OpenSource GitLab code.usgs.gov personal repositories cannot be made public
- Example: https://code.usgs.gov/{user}/{project}.git
Preliminary / Provisional Software Release
This source code can exist in any public USGS GitLab project. Code migrated from a pre-release development software project must receive an administrative security review and Center Director approval prior to migration. Refer to Review for more information on types of reviews.
- Public
- Public visibility is not available by default. Requests to make repositories publicly available should be sent to GS Help Git (gs_help_git@usgs.gov).
- Example: https://code.usgs.gov/{groupname}/{project}.git.
Official Software Release
The official (authoritative) release of source code must exist on USGS OpenSource GitLab. These repositories are marked by Git repository tags indicating that the repository has gone through the required reviews and received approval for release as an official USGS Software Release product. Refer to Review for more information on types of reviews.
- Public
- Public visibility is not available by default. Requests to make repositories publicly available should be sent to GS Help Git (gs_help_git@usgs.gov).
- Example: https://code.usgs.gov/{groupname}/{project}.git.
Start Developing
While actively developing a project, you should follow an "Integration Manager Workflow." Using this technique works especially well when collaboratively developing a project and it works for individual developers as well. This technique encourages using pull requests to integrate code changes. Pull requests facilitate team review and change history tracking as you actively develop software. Recommended references:
Continuous Integration (CI) refers to advanced individual or team methods for integration management. When collaboratively developing a project, each member pulls from and pushes to a remote repository. During this time, not only can the existing code base change, but new code can be added as well as new libraries, and other resources that create dependencies and potential conflicts in the main line (or master branch) of code in the repository. CI ensures when developers submit code to the repository they must first update their code (i.e. rebase it) to reflect the new changes in the repository since they checked out their copy before integration into the main line of code can occur.The GitLab Runner feature is a popular USGS option to implement CI.
