You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+6-4Lines changed: 6 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -790,16 +790,18 @@ The program then writes that one record into a local Parquet file, does a second
790
790
791
791
### Bonus: download a full crawl index and query with DuckDB
792
792
793
-
If you want to run many of these queries, and you have a lot of disk space, you'll want to download the 300 gigabyte index and query it repeatedly. Run:
793
+
In case you want to run many of these queries, and you have a lot of disk space, you'll want to download the 300 gigabyte index and query it repeatedly.
794
+
795
+
> [!IMPORTANT]
796
+
> If you happen to be using the Common Crawl Foundation development server, we've already downloaded these files, and you can run ```make duck_ccf_local_files```
797
+
798
+
To download the crawl index, there are two options: if you have access to the CCF AWS buckets, run:
> If you happen to be using the Common Crawl Foundation development server, we've already downloaded these files, and you can run ```make duck_ccf_local_files```
802
-
803
805
If, by any other chance, you don't have access through the AWS CLI:
0 commit comments