💾 Archived View for gmi.noulin.net › gitRepositories › git-off › file › README.md.gmi captured on 2023-01-29 at 10:53:55. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
README.md (7893B)
1 git-off handles large files in git repos.
2
3
4 - [Introduction](#introduction)
5 - [Install](#install)
6 - [C Version](#c-version)
7 - [Quick Start](#quick-start)
8 - [Other](#other)
9 - [Configuration](#configuration)
10 - [SSH Setup](#ssh-setup)
11 - [Quick ssh and http server setup with docker](#quick-ssh-and-http-server-setup-with-docker)
12 - [Amazon S3](#amazon-s3)
13 - [COMMANDS](#commands)
14 - [Dependencies](#dependencies)
15 - [Platforms](#platforms)
16 - [Hacking](#hacking)
17
18
19 # Introduction
20
21 This is a prototype inspired by [git lfs](https://github.com/git-lfs/git-lfs) and [git fat](https://github.com/jedbrown/git-fat).
22
23 By default the files are stored in the home directory.
24
25 Supported transports are:
26
27 - copy - store locally
28 - rsync
29 - ssh/scp
30 - http (read-only)
31 - s3
32
33 # Install
34
35 ```
36 npm install -g git-off
37 ```
38
39 Then
40
41 ```
42 git off install
43 git off
44 ```
45
46 for help.
47
48 [https://www.npmjs.com/package/git-off](https://www.npmjs.com/package/git-off)
49
50 ## C Version
51
52 There is a git-off written in C in the c directory. To use it do the following step:
53
54 In Debian:
55 ```
56 apt-get install tcc
57 cd /usr/local/bin
58 ln -s $PATH_TO_GITOFF_GIT/c/git-off.c git-off
59 ```
60
61 In MacOS:
62 ```
63 cd /usr/local/bin
64 clang -o git-off $PATH_TO_GITOFF_GIT/c/gitoff.c
65 ```
66
67 Then
68
69 ```
70 git off install
71 git off
72 ```
73
74 for help.
75
76 # Quick Start
77 Setup:
78
79 ```
80 git clone repo
81 git off track '*.bin'
82 git add .
83 git commit
84 git push
85 ```
86
87 # Other
88
89 ```
90 git off install
91 git off mode scp
92 git off scp localhost:/tmp/offStore
93 git off scpuser username
94 git off cc
95 git off ca
96 git off env
97 git off defaults
98 ```
99
100 # Configuration
101
102 The configuration is saved in 3 locations:
103
104 - $GIT_OFF_CONFIG
105 - repo config: .git-off file at the root of repo
106 - global git config
107
108 For each key:
109
110 - git off looks for the value in $GIT_OFF_CONFIG
111 - if the key is not found, git off looks for the value in repo config
112 - if the key is not found, git off looks for the value in the global git config
113
114 Run:
115
116 ```
117 git off env
118 ```
119
120 to see the current configuration.
121
122 The log file location is always configured in the global git config.
123
124 # SSH Setup
125
126 Create an SSH key:
127
128 ```
129 ssh-keygen -t rsa
130 ```
131
132 Copy id_rsa.pub file to SSH server and id_rsa to ~/.ssh
133
134 Configure your private ssh key:
135
136 ```
137 git off pem ~/.ssh/id_rsa
138 ```
139
140 Use git off scp mode:
141
142 ```
143 git off mode scp
144 ```
145
146 Input user, ssh host and path for store:
147
148 ```
149 git off scp $USER@localhost:offStore
150 ```
151
152 Alternatively, setup SSH with ssh config, edit ~/.ssh/config and add your config:
153
154 ```
155 host localhost
156 HostName localhost
157 IdentityFile ~/.ssh/id_rsa
158 User username
159 ```
160
161 # Quick ssh and http server setup with docker
162
163 Start http and ssh servers:
164
165 ```
166 docker run --name gitoffHttp -p 8080:80 -v ~/.git-off/offStore:/usr/share/nginx/html:ro -d nginx:alpine
167 docker run --name gitoffSSH -p 2222:22 -v ~/.git-off/offStore:/root/offStore -d sickp/alpine-sshd
168 ```
169
170 Setup SSH public key:
171
172 ```
173 ssh root@localhost -p 2222 "mkdir .ssh"
174 ```
175
176 The password is __root__
177
178 Setup git off:
179
180 ```
181 git off mode scp
182 git off scp root@localhost:2222/root/offStore
183 ```
184
185 Change root password in SSH server to something more difficult to guess:
186
187 ```
188 ssh root@localhost -p 2222
189 passwd
190 ```
191
192 # Amazon S3
193
194 In AWS S3, create a new bucket for your git off store. Enter the name of the bucket for the bucket option in your S3 store options.
195
196 Select the bucket, and then select Properties. Note the region, and enter the correct region in your S3 store options in git off.
197
198 In AWS IAM, create a new user. Copy the generated key and secret to git off config
199
200 Select your newly created user. In the bottom area, select Permissions > Inline Policy.
201
202 Select Custom Policy.
203
204 To create the custom policy, give it any name you want, and then copy and paste the example policy below. Replace "mybucketname" with your actual bucket name.
205
206 ```
207 {
208 "Version": "2012-10-17",
209 "Statement": [
210 {
211 "Effect": "Allow",
212 "Action": [
213 "s3:ListAllMyBuckets",
214 "s3:ListBucket"
215 ],
216 "Resource": "arn:aws:s3:::*"
217 },
218 {
219 "Action": [
220 "s3:PutObject",
221 "s3:PutObjectAcl",
222 "s3:GetObject",
223 "s3:GetObjectAcl",
224 "s3:DeleteObject",
225 "s3:DeleteObjectAcl"
226 ],
227 "Effect": "Allow",
228 "Resource": [
229 "arn:aws:s3:::mybucketname/*"
230 ]
231 }
232 ]
233 }
234 ```
235
236 Create a credentials file at ~/.aws/credentials:
237
238 ```
239 [default]
240 aws_access_key_id = your_access_key
241 aws_secret_access_key = your_secret_key
242 ```
243
244 Setup s3 in git off:
245
246 ```
247 git off mode s3
248 git off s3region us-west-2
249 git off s3bucket mybucketname
250 ```
251
252 # COMMANDS
253
254 ```
255 git off install [thisrepo]
256 setup git config (default global)
257 thisrepo sets up config in current repo
258 git off mode [thisrepo] [copy|rsync|scp|http|s3]
259 set/show git off mode
260 git off store [thisrepo] [path]
261 set/show git off store path for copy mode
262 git off scp [thisrepo] [host]
263 setup scp config
264 host has format host:path, user@host:path, user@host:port/path
265 Example: localhost:/tmp/offStore
266 git off http [thisrepo] [host]
267 setup http config
268 host has format http://host/path
269 git off curl [thisrepo] [options]
270 setup curl config
271 git off integrity [thisrepo] [enable|disable]
272 set/show git off integrity.
273 when enabled, the SHA of the file received from the store is
274 checked again the SHA of the original file
275 git off pem [thisrepo] [pathToPrivateKey]
276 set/show git off pem.
277 off.pem is the private key for ssh and scp
278 set 'offNoValue' to set an empty value (useful when there are multiple configs)
279 git off sshoptions [thisrepo] [options]
280 set/show git off sshoptions
281 git off scpoptions [thisrepo] [options]
282 set/show git off scpoptions
283 git off rsyncoptions [thisrepo] [options]
284 set/show git off rsyncoptions
285 git off scpuser [thisrepo] [username]
286 setup scp username config
287 git off track
288 setup gitattribute filters
289 example: git off track '*.bin'
290 without parameter, list git off attributes
291 calls git off install
292 git off configAlways [''|GIT_OFF_CONFIG|repo|global]
293 '' disable configAlways
294 GIT_OFF_CONFIG load all configurations from $GIT_OFF_CONFIG
295 repo load all configurations from current git repo
296 global load all configurations from global git config
297 set 'offNoValue' to set an empty value
298 git off s3region [thisrepo] [region]
299 setup amazon s3 region for the bucket
300 git off s3bucket [thisrepo] [bucket]
301 setup amazon s3 bucket
302 git off transform [thisrepo] [enable|disable]
303 enable transform in clean and smudge filters
304 git off transformTo [thisrepo] ['cmd _1 _2']
305 setup transform command for clear filter
306 When the command is empty the regular transport is performed
307 git off transformFrom [thisrepo] ['cmd _1 _2']
308 setup transform command for smudge filter
309 When the command is empty the regular transport is performed
310 git off clean
311 internal filter
312 dont use directly
313 git off pre-push
314 internal filter
315 dont use directly
316 git off smudge
317 internal filter
318 dont use directly
319 git off copyTo [copy|rsync|scp|s3]
320 copy cache to store for specified mode
321 git off push
322 copy cache to store for selected mode
323 git off clearAll
324 delete store, cache and log
325 git off ca
326 delete store, cache and log
327 git off clearCache
328 delete cache in current git
329 git off cc
330 delete cache in current git
331 git off clearStore
332 delete store
333 git off cs
334 delete store
335 git off clearTmp
336 delete tmp in git off cache
337 Useful when transform is enabled
338 git off ct
339 delete tmp in git off cache
340 Useful when transform is enabled
341 git off defaults
342 shows first time config
343 git off env
344 shows config
345 git off help [cmd]
346 git off help. Run git off help command to get help for a specific command.
347 ```
348
349 # Dependencies
350
351 - nodejs
352 - git
353 - rsync for rsync mode
354 - ssh and scp for scp mode
355 - curl for http mode
356
357 # Platforms
358
359 - Linux
360 - MacOS
361
362 # Hacking
363
364 Edit _src/gitoff.coffee_ and then run `./gen.sh`
365
366 Alternatively, edit the generated javascript file directly (_src/git-off_)
367
368 # Contributions
369
370 I'm looking for someone who can contribute to the windows version.