Repositories list

• solrwayback

  Public
  A search interface and wayback machine for the UKWA Solr based warc-indexer framework.

  Java

  •

  Apache License 2.0

  •21•102•70•3•Updated Dec 11, 2024Dec 11, 2024

• netarchivesuite

  Public
  Netarchivesuite development

  Java

  •

  Other

  •23•19•31•21•Updated Nov 28, 2024Nov 28, 2024

• netarchivesuite-docker-compose

  Public
  Testbed for some netarchivesuite docker experiments

  Jinja

  •5•2•0•1•Updated Oct 14, 2024Oct 14, 2024

• crawlrss

  Public
  Crawl RSS - Heritrix 3 add-on

  Java

  •

  Other

  •6•0•0•0•Updated Oct 14, 2024Oct 14, 2024

• heritrix3

  Public
  Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

  Java

  •

  Other

  •762•0•0•0•Updated Oct 14, 2024Oct 14, 2024

• heritrix3-wrapper

  Public
  Small wrapper to start/stop and communicate with Heritrix 3.

  Java

  •

  Apache License 2.0

  •2•3•1•7•Updated Jul 12, 2024Jul 12, 2024

• jwat-tools

  Public
  JWAT Tools

  Java

  •2•5•2•1•Updated Dec 13, 2023Dec 13, 2023

• jwat

  Public
  Java Web Archive Toolkit

  Java

  •2•3•2•3•Updated Dec 13, 2023Dec 13, 2023

• jwarc-cdx-indexer-workflow

  Public
  Will process all warc-files defined in a text file with JWARC and send to a CDX-server (Outback CDX etc.) . If process is stopped and restarted it will continue from where it was.

  Java

  •

  Apache License 2.0

  •0•1•1•0•Updated Dec 7, 2023Dec 7, 2023

• jwat-tools-gui

  Public
  JWAT Tools minimal GUI version

  Java

  •2•0•0•1•Updated Dec 5, 2023Dec 5, 2023

• jwarc

  Public
  Java library for reading and writing WARC files with a typed API

  Java

  •

  Apache License 2.0

  •10•0•0•0•Updated Sep 18, 2023Sep 18, 2023

• webarchive-discovery

  Public
  WARC and ARC indexing and discovery tools.

  Java

  •25•0•3•0•Updated Aug 8, 2023Aug 8, 2023

• cdx-summarize-warc-indexer

  Public
  Summarize Web Archive holdings using an existing SOLR index

  Shell

  •

  Apache License 2.0

  •2•0•0•0•Updated May 17, 2023May 17, 2023

• so-me

  Public
  Social Media harvests

  Shell

  •

  Apache License 2.0

  •0•9•11•0•Updated Jan 11, 2023Jan 11, 2023

• solrwaybackrootproxy

  Public
  Using the solrwaybackrootproxy will improve playback, can redirect and fix leaked resources.

  Java

  •0•1•0•1•Updated Sep 9, 2022Sep 9, 2022

• webdanica

  Public
  System for finding Danish webpages outside the .dk domain

  Java

  •3•1•0•6•Updated Jul 7, 2022Jul 7, 2022

• browsertrix-cloud

  Public
  Danish Royal Library customisations and modifications

  TypeScript

  •

  GNU Affero General Public License v3.0

  •37•0•0•0•Updated Jun 14, 2022Jun 14, 2022

• netsearch

  Public
  Merged search-arctika and search-achon into a multi-module project

  Java

  •2•12•10•6•Updated May 20, 2022May 20, 2022

• heatmap

  Public
  A GitHub-inspired graph for visualising activity

  visualizationvuejsvuejs2+ 1data-visualization

  JavaScript

  •3•32•0•1•Updated Sep 6, 2021Sep 6, 2021

• dvenabler

  Public
  Adds DocValues to Solr index fields without full re-index

  Java

  •

  Apache License 2.0

  •1•9•10•3•Updated Jun 7, 2021Jun 7, 2021

• openwayback-netarkivet-overlay

  Public
  Project to create a customised openwayback for netarkivet using maven overlays.

  Java

  •0•0•0•0•Updated Mar 19, 2021Mar 19, 2021

• openwayback-netarchivesuite

  Public
  NetarchiveSuite fork of OpenWayback

  Java

  •

  Apache License 2.0

  •275•1•0•0•Updated Mar 19, 2021Mar 19, 2021

• vagrant-hadoop-hive-spark

  Public
  Vagrant project to spin up a single node VM running current versions of Hadoop, Hive and Spark

  Shell

  •

  Apache License 2.0

  •59•0•0•0•Updated Dec 7, 2020Dec 7, 2020

• hadoopifications

  Public
  Attempts to create hadoop jobs from other processes

  Java

  •

  Apache License 2.0

  •2•0•0•0•Updated Mar 27, 2020Mar 27, 2020

• logtrix

  Public
  Java library/tool for parsing and summarising Heritrix crawl logs

  Java

  •

  Apache License 2.0

  •1•1•0•0•Updated May 24, 2019May 24, 2019

• umbra

  Public
  A queue-controlled browser automation tool for improving web crawl quality

  Python

  •

  Apache License 2.0

  •21•0•0•0•Updated Apr 23, 2019Apr 23, 2019

• netarchivesuite-umbra-docker

  Public
  Shell

  •1•2•0•0•Updated Mar 29, 2019Mar 29, 2019

• fits-wrapper

  Public
  Small FITS wrapper to run it using a custom classloader and provide some basic JAXB (un)marshalling of the XML output.

  Java

  •

  Apache License 2.0

  •2•0•0•0•Updated Oct 2, 2018Oct 2, 2018

• lap-writer-warc

  Public
  WARC writer for INAs Live Archiving Proxy

  Java

  •1•0•0•0•Updated Aug 31, 2018Aug 31, 2018

• jwat-wayback-resourcestore

  Public
  Wayback resourcestore using JWAT

  Java

  •1•0•0•0•Updated Aug 30, 2018Aug 30, 2018