Git LFS and GitLab Pages - Ansible Software Repository

GitLab: c2platform/phx/examples/git-lfs-and-gitlab-pages 

Welcome to the Git LFS and GitLab Pages example project! This repository demonstrates how to use Git LFS (Large File Storage) and GitLab Pages to create a simple Ansible Software Repository tailored for air-gapped environments. This setup allows hosting and distributing software binaries and Ansible Galaxy collections.

The project is hosted at gitlab.com  and is publicly accessible. The associated GitLab Pages site can be accessed via git-lfs-and-gitlab-pages-c0c5b2.gitlab.io  .

Overview

This repository serves as a proof-of-concept for:

1. Using Git LFS to manage large binary files (e.g., software installers like apache-tomcat-10.1.19.tar.gz).
2. Using GitLab Pages to host and serve these files and Ansible Galaxy collections in a publicly accessible way.
3. Enabling Ansible to download files or collections in air-gapped environments via simple get_url or win_get_url tasks without authentication.
4. Providing a minimalistic implementation of an Ansible Galaxy server for hosting collections as tarball URLs (as defined in requirements.yml).

Important Note: This repository only supports tarball URLs for Ansible Galaxy collections in the requirements.yml file due to the simplicity of the GitLab Pages-based setup. Full Galaxy API functionality is not implemented.

Repository Structure

.
├── apache-tomcat-10.1.19.tar.gz           # Example binary file managed by Git LFS
├── galaxy/                                # Directory hosting Ansible Galaxy collection tarballs
│   ├── ansible.posix-2.0.0.tar.gz
│   ├── ansible.windows-3.1.0.tar.gz
│   ├── c2platform.core-1.0.24.tar.gz
│   ├── c2platform.wincore-1.0.15.tar.gz
│   ├── chocolatey.chocolatey-1.5.3.tar.gz
│   ├── community.crypto-2.26.2.tar.gz
│   ├── community.general-10.7.0.tar.gz
│   ├── community.postgresql-4.1.0.tar.gz
│   └── community.windows-3.0.0.tar.gz
├── index.html                             # GitLab Pages landing page for the repository
├── publish.sh                             # Script to publish content to GitLab Pages
├── README.md                              # Project documentation (this file)
└── requirements.yml                       # Ansible Galaxy collection requirements with tarball URLs

Features

1. Git LFS for Large Files

• Large binary files (e.g., apache-tomcat-10.1.19.tar.gz) are tracked using Git LFS to avoid bloating the repository.
• The .gitattributes file ensures that all .tar.gz files are managed by Git LFS.

2. GitLab Pages for Hosting

• The repository uses GitLab Pages to serve files and collections publicly.
• The CI/CD pipeline (defined in .gitlab-ci.yml) automatically deploys the contents of the repository to GitLab Pages using the publish.sh script.
• Pages access is set to “Everyone with access” to ensure no authentication is required, making it compatible with Ansible tasks like get_url or win_get_url.

3. Ansible Software Repository

• Binary files (e.g., Tomcat) can be downloaded directly via GitLab Pages URLs for use in Ansible playbooks.

• Example usage in an Ansible playbook:

  - name: Download Apache Tomcat
    ansible.builtin.get_url:
      url: https://git-lfs-and-gitlab-pages-c0c5b2.gitlab.io/apache-tomcat-10.1.19.tar.gz 
  
  
      dest: /tmp/apache-tomcat-10.1.19.tar.gz
  

4. Air-Gapped Ansible Galaxy Server

• The galaxy/ directory contains tarballs of Ansible collections that can be used as a minimal Galaxy repository.

• The requirements.yml file specifies direct tarball URLs for collections, which can be used in air-gapped environments.

• Example usage with ansible-galaxy:

  ansible-galaxy collection install -r requirements.yml
  

Setup and Deployment

Prerequisites

• A GitLab account and repository with Git LFS enabled.
• GitLab CI/CD pipelines configured (as provided in .gitlab-ci.yml).
• GitLab Pages enabled for the project with access set to “Everyone with access”.

Deployment Process

• Clone the Repository: Clone this repository to your local machine.
• Push Changes: Any changes pushed to the repository trigger the CI/CD pipeline (defined in .gitlab-ci.yml).
• Publish to GitLab Pages: The publish.sh script syncs the repository content (excluding the public/ directory) to the public/ folder, which is then deployed to GitLab Pages.
• Access Files: Once deployed, files and collections are accessible via URLs like https://git-lfs-and-gitlab-pages-c0c5b2.gitlab.io/<file-path>  .

CI/CD Configuration

The .gitlab-ci.yml file defines a single deploy stage that:

• Uses an Alpine Linux image.
• Installs rsync, git, and git-lfs.
• Pulls large files using git lfs pull.
• Runs publish.sh to prepare content for GitLab Pages.
• Deploys the content with the pages: true directive.

Usage in Ansible

Downloading Binaries

You can download software binaries (e.g., Tomcat) in your Ansible playbooks using get_url or win_get_url:

- name: Download Apache Tomcat
  ansible.builtin.get_url:
    url: https://git-lfs-and-gitlab-pages-c0c5b2.gitlab.io/apache-tomcat-10.1.19.tar.gz 


    dest: /tmp/apache-tomcat-10.1.19.tar.gz

Installing Ansible Collections

Use the provided requirements.yml to install collections from tarball URLs in air-gapped environments:

ansible-galaxy collection install -r requirements.yml

The requirements.yml includes direct URLs to collection tarballs hosted on GitLab Pages, such as:

• https://git-lfs-and-gitlab-pages-c0c5b2.gitlab.io/galaxy/ansible.posix-2.0.0.tar.gz 
• https://git-lfs-and-gitlab-pages-c0c5b2.gitlab.io/galaxy/community.general-10.7.0.tar.gz 

Limitations

• This is a minimalistic implementation of an Ansible Galaxy repository. Only direct tarball URLs are supported in requirements.yml. Advanced Galaxy features (e.g., metadata queries, version resolution) are not available.
• The repository relies on GitLab Pages for hosting, which must be publicly accessible for use without authentication.