todo:
- lockable roots
- full-size pictures for lidl
- metadata language indicator
- image -> images
- separate result processing pipeline for eg. deduplication

depth-first, because this sidesteps any issues with stuff like category IDs being stored in the session rather than the URL


- mark some URLs as 'must redo when recovering from crash' (from the parent URL list), eg. for category pages


----- categories -----

category index		(pagenum 1, categoryIndex true)
	page 2			(pagenum 2, categoryIndex false)
	page 1			(pagenum 1, categoryIndex false)
	page 3			(pagenum 3, categoryIndex false)
	page 4			(pagenum 4, categoryIndex false)