anonderpling commited on
Commit
d99422e
1 Parent(s): 2d934ba

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -9
README.md CHANGED
@@ -1,11 +1,8 @@
1
- # A mirror of anything I find interesting on civitai or elsewhere on the web.
2
-
3
- I dont usually download character loras, so look
4
- elsewhere for those.
5
 
6
  This repo is *mostly* organized around the structure
7
  that's necessary to import the entire repo into an
8
- automatic1111 install. This is because models were
9
  uploaded directly from such an install via google drive in
10
  a colab session.
11
 
@@ -14,7 +11,7 @@ https://hf.co/anonderpling/repo_uploader, however I expect
14
  to go back to a real file system and git uploads soon,
15
  hopefully paperspace can help with that.
16
 
17
- ## My workflow
18
 
19
  My workflow for downloading files into paperspace gradient
20
  is to download the entire repo *without* pulling LFS
@@ -24,7 +21,9 @@ workflow should work with colab, too. Whether you use
24
  colab or paperspace, you'll probably need the latest
25
  version of git to use `sparse-checkout --add`.
26
 
27
- ```
 
 
28
  !GIT_LFS_SPARSE_CHECKOUT=1 git clone git@hf.co:anonderpling/civitai_mirror # this is my command so I can push changes, you'll need to use the https://hf.co/ instead of git@hf.co:
29
  !cd civitai_mirror
30
  !git sparse-checkout set embeddings # embeddings are small, so it's easy enough to just pull all of them
@@ -43,9 +42,9 @@ version of git to use `sparse-checkout --add`.
43
  !aria2 -x16 --split=16 -i aria2.in.txt
44
  ```
45
 
46
- to upload more files:
47
 
48
- ```
49
  # enable the git lfs filters
50
  !pip install huggingface_hub
51
  !huggingface-cli lfs_enable_largefiles .
@@ -69,6 +68,39 @@ print('https://hf.co/settings/keys')
69
  !git push
70
  ```
71
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
  #### It's extremely important to remove the ssh key from your HF repo after you're done with it. this ensures that nobody else can access your account.
73
 
74
  Paperspace makes free notebooks public, and I'm not sure
@@ -95,3 +127,9 @@ afaik)
95
  2. move sfw models into a subdirectory
96
  3. consider moving locons to their own directory in models, now that I'm using paperspace...
97
  - Perpetual: keep an eye on civitai update notifications
 
 
 
 
 
 
 
1
+ # A mirror of anything interesting on civitai or elsewhere on the web.
 
 
 
2
 
3
  This repo is *mostly* organized around the structure
4
  that's necessary to import the entire repo into an
5
+ automatic1111 install. This is because models were originally
6
  uploaded directly from such an install via google drive in
7
  a colab session.
8
 
 
11
  to go back to a real file system and git uploads soon,
12
  hopefully paperspace can help with that.
13
 
14
+ ## Workflow
15
 
16
  My workflow for downloading files into paperspace gradient
17
  is to download the entire repo *without* pulling LFS
 
21
  colab or paperspace, you'll probably need the latest
22
  version of git to use `sparse-checkout --add`.
23
 
24
+ ## How to upload and download files
25
+
26
+ ```bash
27
  !GIT_LFS_SPARSE_CHECKOUT=1 git clone git@hf.co:anonderpling/civitai_mirror # this is my command so I can push changes, you'll need to use the https://hf.co/ instead of git@hf.co:
28
  !cd civitai_mirror
29
  !git sparse-checkout set embeddings # embeddings are small, so it's easy enough to just pull all of them
 
42
  !aria2 -x16 --split=16 -i aria2.in.txt
43
  ```
44
 
45
+ to upload more files via git:
46
 
47
+ ```bash
48
  # enable the git lfs filters
49
  !pip install huggingface_hub
50
  !huggingface-cli lfs_enable_largefiles .
 
68
  !git push
69
  ```
70
 
71
+ to upload a single file via huggingface_hub:
72
+
73
+ ```python
74
+ inurl=api.upload_folder( # https://huggingface.co/docs/huggingface_hub/v0.14.1/en/package_reference/hf_api#huggingface_hub.HfApi.upload_folder
75
+ token="hf_token", # only needed if you didn't already use huggingface_hub.login() previously.
76
+ path_or_fileobj="/content/stable-diffusion-webui/models/Stable-diffusion/Rev-Animated.safetensors", # location of the file you want to upload
77
+ path_in_repo="models/Stable-diffusion/Rev-Animated.safetensors",
78
+ repo_id="mirroring/civitai_mirror",
79
+ commit_message="Uploading Rev Animated", # optional.
80
+ commit_description="I want to upload Rev Animated because it's a special file for me.\nPlease accept my PR. I don't want to host it on my own HF repo!" # optional. Required (here) for PRs.
81
+ create_pr=True # optional. needed if you're not a special person allowed to add new files to the repo (ie, if you just want us to mirror something; make sure to fill out the description/message above, as well)
82
+ )
83
+ print("Uploaded files. Check them out at "+inurl)
84
+ ```
85
+
86
+ to upload multiple files via huggingface_hub:
87
+
88
+ ```python
89
+ inurl=api.upload_folder( # https://huggingface.co/docs/huggingface_hub/v0.14.1/en/package_reference/hf_api#huggingface_hub.HfApi.upload_folder
90
+ token="hf_token", # only needed if you didn't already use huggingface_hub.login() previously.
91
+ folder_path="/stable-diffusion-webui/models/",
92
+ path_in_repo="models",
93
+ repo_id="mirroring/civitai_mirror",
94
+ allow_patterns="*.safetensors", # optional. Only upload certain files.
95
+ ignore_patterns=["*.tmp","tmp/*","*.jpg"], # optional. Ignore certain files.
96
+ commit_message="Uploading my own models",
97
+ commit_description="I want to upload these models because..."
98
+ create_pr=True # needed if you're not a special person allowed to add new files to the repo (ie, if you just want us to mirror something)
99
+ )
100
+ print("Uploaded files. Check them out at "+inurl)
101
+ ```
102
+
103
+
104
  #### It's extremely important to remove the ssh key from your HF repo after you're done with it. this ensures that nobody else can access your account.
105
 
106
  Paperspace makes free notebooks public, and I'm not sure
 
127
  2. move sfw models into a subdirectory
128
  3. consider moving locons to their own directory in models, now that I'm using paperspace...
129
  - Perpetual: keep an eye on civitai update notifications
130
+
131
+ ## License
132
+
133
+ I like WTFPL. That means anything generated by @anonderpling is licensed WTFPL. That means pretty much only this readme, since everything else is someone else's work.
134
+
135
+ Other files? You'll have to find their official sources to find their licenses. The licenses for the civitai uploads *might* be in the .civitai.info files, which are standard json (AFAIK) as returned by the civitai API.