idk/Git_over_I2P
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

aaaaand... github abuse

Browse Source
This commit is contained in:
idk
2020-03-18 19:49:50 -04:00
parent 5021cbaae8
commit 69d7242116
5 changed files with 425 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
Makefile Normal file
View File

@ -0,0 +1,14 @@
install:
mkdir -p $(HOME)/Workspace/mtn/i2p.www/i2p2www/blog/2020/03/16
cp gitlab-over-i2p.rst $(HOME)/Workspace/mtn/i2p.www/i2p2www/blog/2020/03/16
#cp *.png $(HOME)/Workspace/mtn/i2p.www/i2p2www/static/images/git/
rst:
pandoc BUNDLES.md -o bundle-over-snark.rst
upload-bundle:
cat desc | gothub release -p -u eyedeekay -r git-over-i2p -t bundles -n "Pregenerated Bundles" -d -; true
cat desc | gothub edit -p -u eyedeekay -r git-over-i2p -t bundles -n "Pregenerated Bundles" -d -; true
gothub upload -R -u eyedeekay -r git-over-i2p -t bundles -n "i2p.i2p.bundle.torrent" -f $(HOME)/.i2p/i2psnark/i2p.i2p.bundle.torrent
gothub upload -R -u eyedeekay -r git-over-i2p -t bundles -n "i2p.i2p.bundle" -f $(HOME)/.i2p/i2psnark/i2p.i2p.bundle

111
bundle-over-snark.rst Normal file
View File

@ -0,0 +1,111 @@
Using a git bundle to fetch the I2P source code
===============================================
Cloning large software repositories over I2P can be difficult, and using
git can sometimes make this harder. Fortunately, it can also sometimes
make it easier. Git has a ``git bundle`` command which can be used to
turn a git repository into a file which git can then clone, fetch, or
import from a location on your local disk. By combining this capability
with bittorrent downloads, we can solve our remaining problems with
``git clone``.
Before you Start
----------------
If you intend to generate a git bundle, you **must** already possess a
full copy of the **git** repository, not the mtn repository. You can get
it from github or from git.idk.i2p, but a shallow clone(a clone done to
–depth=1) *will not* *work*. It will fail silently, creating what looks
like a bundle, but when you try to clone it it will fail. If you are
just retrieving a pre-generated git bundle, then this section does not
apply to you.
Fetching I2P Source via Bittorrent
----------------------------------
Someone will need to supply you with a torrent file or a magnet link
corresponding to an existing ``git bundle`` that they have already
generated for you. A recent, correctly-generated bundle of the mainline
i2p.i2p source code as-of Wednesday, March 18, 2020, can be found inside
of I2P at my pastebin
`paste.idk.i2p/f/4hq37i <http://paste.idk.i2p/f/4h137i>`__.
Once you have a bundle, you will need to use git to create a working
repository from it. If you’re using GNU/Linux and i2psnark, the git
bundle should be located in $HOME/.i2p/i2psnark or, as a service on
Debian, /var/lib/i2p/i2p-config/i2psnark. If you are using BiglyBT on
GNU/Linux, it is probably at “$HOME/BiglyBT Downloads/” instead. The
examples here assume I2PSnark on GNU/Linux, if you use something else,
replace the path to the bundle with the download directory preferred by
your client and platform.
Using ``git clone``
~~~~~~~~~~~~~~~~~~~
Cloning from a git bundle is easy, just:
::
git clone $HOME/.i2p/i2psnark/i2p.i2p.bundle
If you get the following error, try using git init and git fetch
manually instead.
::
fatal: multiple updates for ref 'refs/remotes/origin/master' not allowed
Using ``git init`` and ``git fetch``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First, create an i2p.i2p directory to turn into a git repository.
::
mkdir i2p.i2p && cd i2p.i2p
Next, initialize an empty git repository to fetch changes back into.
::
git init
Finally, fetch the repository from the bundle.
::
git fetch $HOME/.i2p/i2psnark/i2p.i2p.bundle
Replace the bundle remote with the upstream remote
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now that you have a bundle, you can keep up with changes by setting the
remote to the upstream repository source.
::
git remote set-url origin git@127.0.0.1:i2p-hackers/i2p.i2p
Generating a Bundle
-------------------
First, follow the `Git guide for Users <GIT.md>`__ until you have a
successfully ``--unshallow``\ ed clone of clone of the i2p.i2p
repository. If you already have a clone, make sure you run
``git fetch --unshallow`` before you generate a torrent bundle.
Once you have that, simply run the corresponding ant target:
::
ant bundle
and copy the resulting bundle into your I2PSnark downloads directory.
For instance:
::
cp i2p.i2p.bundle* $HOME/.i2p/i2psnark/
In a minute or two, I2PSnark will pick up on the torrent. Click on the
“Start” button to begin seeding the torrent.

16
desc Normal file
View File

@ -0,0 +1,16 @@
Using these files:
==================
These files are provided so that people who wish to can help seed the torrent.
To use these files, download them to your i2psnark directory:
First download the file itself:
```wget -c -O $HOME/.i2p/i2psnark/i2p.i2p.bundle https://github.com/eyedeekay/git-over-i2p/releases/download/bundles/i2p.i2p.bundle```
Then download the torrent.
```wget -c -O $HOME/.i2p/i2psnark/i2p.i2p.bundle.torrent https://github.com/eyedeekay/git-over-i2p/releases/download/bundles/i2p.i2p.bundle.torrent```
I2PSnark will pick up the torrent momentarily. Click the "Start" button to help
seed the torrent.

251
gitlab-over-i2p.rst Normal file
View File

@ -0,0 +1,251 @@
{% trans -%}Gitlab over I2P Setup{%- endtrans %}
================================================
.. meta::
:author: idk
:date: 2020-03-16
:excerpt: {% trans -%}Mirror I2P Git repositories and Bridge Clearnet repositories for others.{%- endtrans %}
{% trans -%}
This is the setup process I use for configuring Gitlab and I2P, with
Docker in place to manage the service itself. Gitlab is very easy to
host on I2P in this fashion, it can be administered by one person
without much difficulty. In my configuration, I use a Debian VM to host
docker containers and an I2P router, on a Debian Host system, however,
this may be more than necessary for some people. These instructions
should work on any Debian-based system, regardless of whether it is in a
VM or not, and should easily translate to any system where Docker and an
I2P router are available. This guide starts at Docker and does not
assume any VM underneath.
{%- endtrans %}
{% trans -%}Dependencies and Docker{%- endtrans %}
---------------------------------------------------
{% trans -%}
Because Gitlab runs in a container, we only need to install the
dependencies required for the container on our main system.
Conveniently, you can install everything you need with:
{%- endtrans %}
::
sudo apt install docker.io
{% trans -%}
on an otherwise unmodified Debian system, or if you have added Docker’s
own “Community” Debian repository, you may use:
{%- endtrans %}
::
sudo apt install docker-ce
{% trans -%}
instead.
{%- endtrans %}
------------------------------------------------------
{% trans -%}Fetch the Docker Containers{%- endtrans %}
------------------------------------------------------
{% trans -%}
Once you have docker installed, you can fetch the docker containers
required for gitlab. *Don’t run them yet.*
{%- endtrans %}
::
docker pull gitlab/gitlab-ce
{% trans -%}
For those who are concerned, the gitlab-ce Docker image is built using
only Ubuntu Docker images as a parent, which are built from scratch
images. Since there are no third-party images involved here, updates
should come as soon as they are available in the host images. To review
the Dockerfile for yourself, visit the `Gitlab source
code <https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/docker/Dockerfile>`__.
{%- endtrans %}
{% trans -%}Set up an I2P HTTP Proxy for Gitlab to use(Important information, optional steps){%- endtrans %}
------------------------------------------------------------------------------------------------------------
{% trans -%}
Gitlab servers inside of I2P can be run with or without the ability to
interact with servers on the internet outside of I2P. In the case where
the Gitlab server is *not allowed* to interact with servers outside of
I2P, they cannot be de-anonymized by cloning a git repository from a git
server on the internet outside of I2P. They can, however, export and
mirror repositories from other git services inside of I2P.
{%- endtrans %}
{% trans -%}
In the case where the Gitlab server is *allowed* to interact with
servers outside of I2P, it can act as a “Bridge” for the users, who can
use it to mirror content outside I2P to an I2P-accessible source,
however it *is not anonymous* in this case. Git services on the
non-anonymous internet will be connected to directly.
{%- endtrans %}
{% trans -%}
**If you want to have a bridged, non-anonymous Gitlab instance with
access to** **web repositories,** no further modification is necessary.
{%- endtrans %}
{% trans -%}
**If you want to have an I2P-Only Gitlab instance with no access to
Web-Only** **Repositories**, you will need to configure Gitlab to use an
I2P HTTP Proxy. Since the default I2P HTTP proxy only listens on
``127.0.0.1``, you will need to set up a new one for Docker that listens
on the Host/Gateway address of the Docker network, which is usually
``172.17.0.1``. I configure mine on port ``4446``.
{%- endtrans %}
{% trans -%}Start the Container Locally{%- endtrans %}
------------------------------------------------------
{% trans -%}
Once you have that set up, you can start the container and publish your
Gitlab instance locally.
{%- endtrans %}
::
docker run --detach \
--env HTTP_PROXY=http://172.17.0.1:4446 \
--publish 127.0.0.1:8443:443 --publish 127.0.0.1:8080:80 --publish 127.0.0.1:8022:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab:Z \
--volume /srv/gitlab/logs:/var/log/gitlab:Z \
--volume /srv/gitlab/data:/var/opt/gitlab:Z \
gitlab/gitlab-ce:latest
{% trans -%}
Visit your Local Gitlab instance and set up your admin account. Choose a
strong password, and configure user account limits to match your
resources.
{%- endtrans %}
{% trans -%}Modify gitlab.rb(Optional, but a good idea for “Bridged non-anonymous” hosts){%- endtrans %}
--------------------------------------------------------------------------------------------------------
{% trans -%}
It’s also possible to apply your HTTP Proxy settings in a more granular
way, so that you can only allow users to mirror repositories from the
domains that you choose. Since the domain is presumably operated by an
organization, you can use this to ensure that repositories that are
mirrorable follow a reasonable set of policies. After all, there is far
more abusive content on the non-anonymous internet than there is on I2P,
we wouldn’t want to make it too easy to introduce abusive content from
such a nefarious place.
{%- endtrans %}
{% trans -%}
Add the following lines to your gitlab.rb file inside the
/src/gitlab/config container. These settings will take effect when you
restart in a moment.
{%- endtrans %}
::
gitlab_rails['env'] = {
"http_proxy" => "http://172.17.0.1:4446",
"https_proxy" => "http://172.17.0.1:4446",
"no_proxy" => ".github.com,.gitlab.com"
}
gitaly['env'] = {
"http_proxy" => "http://172.17.0.1:4446",
"https_proxy" => "http://172.17.0.1:4446",
"no_proxy" => "unix,.github.com,.gitlab.com"
}
gitlab_workhorse['env'] = {
"http_proxy" => "http
"https_proxy" => "http://172.17.0.1:4446",
"no_proxy" => "unix,.github.com,.gitlab.com"
}
---------------------------------------------------------------------------------
{% trans -%}Set up your Service tunnels and sign up for a Hostname{%- endtrans %}
---------------------------------------------------------------------------------
{% trans -%}
Once you have Gitlab set up locally, go to the I2P Router console. You
will need to set up two server tunnels, one leading to the Gitlab
web(HTTP) interface on TCP port 8080, and one to the Gitlab SSH
interface on TCP Port 8022.
{%- endtrans %}
{% trans -%}Gitlab Web(HTTP) Interface{%- endtrans %}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
{% trans -%}
For the Web interface, use an “HTTP” server tunnel. From
http://127.0.0.1:7657/i2ptunnelmgr launch the “New Tunnel Wizard” and
enter the following values at each step:
{%- endtrans %}
1. {% trans -%}Select “Server Tunnel”{%- endtrans %}
2. {% trans -%}Select “HTTP Server”{%- endtrans %}
3. {% trans -%}Fill in “Gitlab Web Service” or otherwise describe the tunnel{%- endtrans %}
4. {% trans -%}Fill in ``127.0.0.1`` for the host and ``8080`` for the port.{%- endtrans %}
5. {% trans -%}Select “Automatically start tunnel when Router Starts”{%- endtrans %}
6. {% trans -%}Confirm your selections.{%- endtrans %}
{% trans -%}Register a Hostname(Optional){%- endtrans %}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
{% trans -%}
Web services on I2P can register hostnames for themselves by providing
an authentication string to a jump service provider like
`stats.i2p <http://stats.i2p>`__. To do this, open the
http://127.0.0.1:7657/i2ptunnelmgr again and click on the “Gitlab Web
Service” item you just set up. Scroll to the bottom of the “Edit Server
Settings” section and click Registration Authentication. Copy the field
that says “Authentication for adding Hostname” and visit
`stats.i2p <http://stats.i2p/i2p/addkey.html>`__ to add your hostname.
Note that if you want to use a subdomain(Like git.idk.i2p) then you will
have to use the correct authentication string for your subdomain, which
is a little bit more complicated and merits it’s own instructions.
{%- endtrans %}
{% trans -%}Gitlab SSH Interface{%- endtrans %}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
{% trans -%}
For the SSH interface, use a “Standard” server tunnel. From
http://127.0.0.1:7657/i2ptunnelmgr launch the “New Tunnel Wizard” and
enter the following values at each step:
{%- endtrans %}
1. {% trans -%}Select “Server Tunnel”{%- endtrans %}
2. {% trans -%}Select “Standard Server”{%- endtrans %}
3. {% trans -%}Fill in “Gitlab SSH Service” or otherwise describe the tunnel{%- endtrans %}
4. {% trans -%}Fill in ``127.0.0.1`` for the host and ``8022`` for the port.{%- endtrans %}
5. {% trans -%}Select “Automatically start tunnel when Router Starts”{%- endtrans %}
6. {% trans -%}Confirm your selections.{%- endtrans %}
{% trans -%}Re-start the Gitlab Service with the new Hostname{%- endtrans %}
----------------------------------------------------------------------------
{% trans -%}
Finally, if you either modified ``gitlab.rb`` or you registered a
hostname, you will need to re-start the gitlab service for the settings
to take effect.
{%- endtrans %}
::
docker stop gitlab
docker rm gitlab
docker run --detach \
--hostname your.hostname.i2p \
--hostname thisisreallylongbase32hostnamewithfiftytwocharacters.b32.i2p \
--env HTTP_PROXY=http://172.17.0.1:4446 \
--publish 127.0.0.1:8443:443 --publish 127.0.0.1:8080:80 --publish 127.0.0.1:8022:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab:Z \
--volume /srv/gitlab/logs:/var/log/gitlab:Z \
--volume /srv/gitlab/data:/var/opt/gitlab:Z \
gitlab/gitlab-ce:latest

33
resources/07-Git Repo-i2ptunnel.config Normal file
View File

@ -0,0 +1,33 @@
# NOTE: This I2P config file must use UTF-8 encoding
# Last saved: Jan 7, 2020, 9:38 PM
description=SSH Tunnel for git.idk.i2p
interface=127.0.0.1
listenPort=7822
name=Git Repo
option.i2cp.closeIdleTime=1800000
option.i2cp.closeOnIdle=false
option.i2cp.delayOpen=false
option.i2cp.destination.sigType=7
option.i2cp.newDestOnResume=false
option.i2cp.reduceIdleTime=1200000
option.i2cp.reduceOnIdle=false
option.i2cp.reduceQuantity=1
option.i2p.streaming.connectDelay=500
option.i2p.streaming.maxWindowSize=16
option.inbound.backupQuantity=3
option.inbound.length=1
option.inbound.lengthVariance=0
option.inbound.nickname=Git Repo
option.inbound.quantity=3
option.outbound.backupQuantity=3
option.outbound.length=1
option.outbound.lengthVariance=0
option.outbound.nickname=Git Repo
option.outbound.quantity=3
option.persistentClientKey=true
option.useSSL=false
privKeyFile=i2ptunnel6.1-privKeys.dat
sharedClient=false
startOnLoad=true
targetDestination=redactedfromthisversionsetupanotheritseasy.b32.i2p
type=client