any pointers to sites / papers / articles which cover the "deep web" ?
Any work done to make the deep web via an api / interface accessible for search / meta engines (like smbmeta.org BTW is this project dead ?). I think about a interface which allows spiders to "download" or sync a file which contains search meta data / information (e.g. keywords, words and corresponding URLs, i.e. an index) from the search engine / CMS / DMS on the site (only the pages / files / data which the owner of the site wants to be publish or publicly available for an external search engine) ?