Записывает URL запроса Firefox на стандартный вывод или в файл?

Question

Мне нужно получить журнал всех файлов, извлеченных из Интернета, когда я загружаю страницу в Firefox; Я действительно не могу использовать Firefox Web Developer (Инструменты) / Сетевой монитор, потому что некоторые страницы, которые используют Flash, имеют тенденцию вызывать сбой Firefox для меня, когда у меня открыт Сетевой монитор (даже если они, похоже, загружаются правильно без сетевого монитора). Поэтому мне нужно получить список URL-адресов, которые "выживут" в таких сбоях.

Мне известно, что я могу получить журнал сетевого трафика из Firefox, используя переменные среды, например, в Linux (это работает в Firefox 43, но я не смог заставить его работать с MOZ_LOG_MODULES/MOZ_LOG_FILE):

NSPR_LOG_MODULES=timestamp,nsHttp:3,sync NSPR_LOG_FILE=/tmp/ff.log /path/to/firefox/firefox

Однако, это печатает очень подробную информацию с полными HTTP-запросами / ответами, такими как:

2017-01-07 19:18:49.799922 UTC - -1371059392[b726e9c0]: Http2Session::ALPNCallback version=303
2017-01-07 19:18:49.981528 UTC - -1220458752[b726e0c0]: http request [
2017-01-07 19:18:49.981551 UTC - -1220458752[b726e0c0]:   POST / HTTP/1.1
2017-01-07 19:18:49.981560 UTC - -1220458752[b726e0c0]:   Host: ocsp.int-x3.letsencrypt.org
2017-01-07 19:18:49.981567 UTC - -1220458752[b726e0c0]:   User-Agent: Mozilla/5.0 (X11; Linux i686; rv:4
3.0) Gecko/20100101 Firefox/43.0
2017-01-07 19:18:49.981574 UTC - -1220458752[b726e0c0]:   Accept: text/html,application/xhtml+xml,applic
ation/xml;q=0.9,*/*;q=0.8
2017-01-07 19:18:49.981580 UTC - -1220458752[b726e0c0]:   Accept-Language: en-US,en;q=0.5
2017-01-07 19:18:49.981589 UTC - -1220458752[b726e0c0]:   Accept-Encoding: gzip, deflate
2017-01-07 19:18:49.981595 UTC - -1220458752[b726e0c0]:   DNT: 1
2017-01-07 19:18:49.981604 UTC - -1220458752[b726e0c0]:   Content-Length: 85
2017-01-07 19:18:49.981611 UTC - -1220458752[b726e0c0]:   Content-Type: application/ocsp-request
2017-01-07 19:18:49.981617 UTC - -1220458752[b726e0c0]:   Connection: keep-alive
2017-01-07 19:18:49.981639 UTC - -1220458752[b726e0c0]: ]
2017-01-07 19:18:50.071135 UTC - -1371059392[b726e9c0]: http response [
2017-01-07 19:18:50.071174 UTC - -1371059392[b726e9c0]:   HTTP/1.1 200 OK
2017-01-07 19:18:50.071195 UTC - -1371059392[b726e9c0]:   Server: nginx
2017-01-07 19:18:50.071203 UTC - -1371059392[b726e9c0]:   Content-Type: application/ocsp-response
2017-01-07 19:18:50.071210 UTC - -1371059392[b726e9c0]:   Content-Length: 527
2017-01-07 19:18:50.071218 UTC - -1371059392[b726e9c0]:   Etag: "3FFFC13100849446EA3D8FA68B0077AF4792776825416386C5CF76104592E0CE"
2017-01-07 19:18:50.071225 UTC - -1371059392[b726e9c0]:   Last-Modified: Wed, 04 Jan 2017 00:00:00 UTC
2017-01-07 19:18:50.071233 UTC - -1371059392[b726e9c0]:   Cache-Control: public, no-transform, must-revalidate, max-age=24762
2017-01-07 19:18:50.071240 UTC - -1371059392[b726e9c0]:   Expires: Sun, 08 Jan 2017 02:11:32 GMT
2017-01-07 19:18:50.071247 UTC - -1371059392[b726e9c0]:   Date: Sat, 07 Jan 2017 19:18:50 GMT
2017-01-07 19:18:50.071254 UTC - -1371059392[b726e9c0]:   Connection: keep-alive
2017-01-07 19:18:50.071259 UTC - -1371059392[b726e9c0]: ]

... и мне действительно нужны только полные URL, что-то вроде списков, отображаемых сетевым монитором Firefox:

Так есть ли способ убедить Firefox вывести в реальном времени только stdout или в файл только те URL-адреса, к которым получен доступ, чтобы в случае сбоя Firefox у меня все еще оставался журнал доступа к URL-адресам?

Answer 1

Мне нужно получить список URL-адресов, которые "переживут" такие сбои.

Появится дополнение Firefox для HTTP HTTP Logger, которое будет соответствовать вашим потребностям:

Записывает HTTP-запросы (реферер, HTTP-метод и URL) в текстовый файл по соображениям безопасности и аудита.

Регистратор исходного HTTP-запроса

Иногда может быть полезно регистрировать каждый URL-адрес, доступ к которому осуществляется через веб-браузер Mozilla Firefox. Это может быть связано с безопасностью, аудитом или другими причинами, по которым вы хотите регистрировать доступные URL. Я разработал дополнение HTTP Request Logger для отслеживания и обнаружения вредоносной веб-страницы, которая вызывала уязвимость CVE-2010-4452 в Oracle Java через специально созданный тег апплета и соответствующий класс апплета Java. Вы можете скачать и установить дополнение отсюда: HTTP Request Logger. Вы также можете просмотреть проект на GitHub: prekageo/http-request-logger.

Надстройка реализует компонент JavaScript XPCOM. Этот компонент зарегистрирован в chrome.manifest. Компонент регистрирует наблюдателя для тем http-on-modify-request. Функция обратного вызова наблюдателя вызывается, когда новый HTTP-запрос готов к отправке. Функция добавляет в простой текстовый файл, расположенный на рабочем столе пользователя, строку, состоящую из реферера, метода HTTP и URL-адреса HTTP-запроса.

Исходный модуль HTTP Request Logger для Mozilla Firefox

Я не знаю, выживет ли выход из строя Firefox.

Вывод записывается в http-request-log.txt на рабочем столе (полное имя C:\Users\username\Desktop\http-request-log.txt

Пример вывода (после перезагрузки https://superuser.com/questions):

(none) GET https://superuser.com/questions
(none) POST http://clients1.google.com/ocsp
https://superuser.com/questions GET https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js
https://superuser.com/questions GET https://cdn.sstatic.net/Js/stub.en.js?v=5d747a3197db
https://superuser.com/questions GET https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea
https://superuser.com/questions GET https://www.gravatar.com/avatar/1c46f449aee8300e9bcf2e6462e0127c?s=32&d=identicon&r=PG
https://chat.stackexchange.com/rooms/114/ask-a-super-user-moderator POST https://chat.stackexchange.com/events
https://superuser.com/questions GET https://i.stack.imgur.com/Ohyu2.png?s=48&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/adab46b83f2276ff0a2da05c3aea1a1e?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/49427fda382a6ebf890c6401eb1dc117?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/8c7bfdd599703d0f59b8adb35c33384d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/ce3dd556aaf605869d870670f4addad2?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/b79191086ccc83d2efbf0346a9498552?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/2b4de46accb1b608fc45963de8c171d8?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/b61ea906fab08361132d0f98ffb89906?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://i.stack.imgur.com/pw4Ug.gif?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/93ec7cff90c748bc4f4da0c540e68bfc?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/5df3eeb6bd2ca404e3b21a08721ad45d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/c6bac24248317129b390d60b7c088e40?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/1bcd453599a20c9eb444902d81c827c0?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/2e6a415fd9d93e089c5ac7138b306c8d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/VyvYN.png?s=32&g=1
https://superuser.com/questions GET https://lh5.googleusercontent.com/-snn4nMpy9Pw/AAAAAAAAAAI/AAAAAAAAATY/f_fJpek-wHo/photo.jpg?sz=32
https://superuser.com/questions GET https://i.stack.imgur.com/xObp1.png?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/994136ff5c5a426f77ded906bc1323b0?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/98CKO.png?s=32&g=1
https://superuser.com/questions GET https://graph.facebook.com/867261659995682/picture?type=large
https://superuser.com/questions GET https://i.stack.imgur.com/oGmjT.png?s=32&g=1
https://superuser.com/questions GET https://i.stack.imgur.com/gaAXg.jpg?s=32&g=1
https://superuser.com/questions GET https://lh6.googleusercontent.com/-EwJ5_a_MCkc/AAAAAAAAAAI/AAAAAAAAAD8/Hra648Whrto/photo.jpg?sz=32
https://superuser.com/questions GET https://www.gravatar.com/avatar/bb3b38b5e3dac0b2df433d5c78cf9fc5?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/763da2b58630f974d468d88c26ac2e4e?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/c273f9992292a61959df77bca08665a3?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/41e856251e65f04dd94d72eed93e48e6?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/bc9f58c57918b259c487410155bd9e23?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/7db005472dc422ca5cbb143a5fd36125?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/b08fc2b0be91884d666ef12eac8d0d8f?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/6d1f8e5a125a589884b3784caa11736b?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://i.stack.imgur.com/ORos2.png?s=32&g=1
https://superuser.com/questions GET https://lh3.googleusercontent.com/-2DRzHlFuo4A/AAAAAAAAAAI/AAAAAAAAAJ4/UoePoxsDseg/photo.jpg?sz=32
https://superuser.com/questions GET https://i.stack.imgur.com/cYO05.jpg?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/bb7cd698db4bb8e9fa866d5154a05355?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/f9c738932874403fd70f2c617951f9e2?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/Dqrx8.png?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/fc34739487a5e7e115acdbadf95c8fb3?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://lh4.googleusercontent.com/-Bah9vRlCwX8/AAAAAAAAAAI/AAAAAAAAAF4/sHjIDOHJT-c/photo.jpg?sz=32
https://superuser.com/questions GET https://www.gravatar.com/avatar/01cc31c9b124d090aeb3e06f94bb8ed8?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/417ffb01cabc10df5dc2ffad281e6a65?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/52eda3f9c34ef88ca164606301366366?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/a7da99412e7177afd0951d7511bdfec8?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/b3ac28f02a59435e921d244c0d82fbbf?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/e880c06c3667fe7c282f0dcb22e48525?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/f7a735cfc6d5fdf3ef44bdbb94fe3beb?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/7ebac3a51e6820a973e14c5b8b28e452?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/1859855084a2f798e294422cf6ba2d31?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/ee06ba399c5eaf3f5a337a96207c2768?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://graph.facebook.com/100000616410966/picture?type=large
https://superuser.com/questions GET https://engine.adzerk.net/ados.js
https://superuser.com/questions GET https://www.google-analytics.com/analytics.js
https://superuser.com/questions GET https://secure.quantserve.com/quant.js
https://superuser.com/questions GET https://sb.scorecardresearch.com/beacon.js
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/img/share-sprite-new.svg?v=78be252218f3
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/Sites/superuser/img/sprites.svg?v=6f5b3bbd35ac
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/img/favicons-sprite16.png?v=41af8148f8e0
(none) GET http://i.stack.imgur.com/05rE1.png
https://superuser.com/questions GET https://cdn.rawgit.com/ofirdagan/cross-domain-local-storage/d779a81a6383475a1bf88595a98b10a8bd5bb4ae/dist/scripts/xdLocalStorage.min.js
https://superuser.com/questions GET https://cdn.sstatic.net/Js/full.en.js?v=53cad7e6923d
https://superuser.com/questions GET https://scontent.xx.fbcdn.net/v/t1.0-1/p200x200/15621850_1188627624525749_7199393790071419683_n.jpg?oh=e4fa46a17dc0d6b646a664641891f18b&oe=58D7DB42
https://superuser.com/questions GET https://scontent.xx.fbcdn.net/v/t1.0-1/p200x200/14141922_1247416555288880_9079557968728006677_n.jpg?oh=05088f1d9fc7ee9e8c50a2795c018cb2&oe=59132CC4
https://superuser.com/questions GET https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css
(none) GET https://i.stack.imgur.com/05rE1.png
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20json%20where%20url%3D%22http%3A%2F%2Fstackexchange.com%2Fhot-questions-for-mobile%22&format=json
https://superuser.com/questions GET https://api.stackexchange.com/2.2/users/607889;194121;683117;682924;662836;618039;683114;683118;593210;3365;683110;529857;683096;683101;462324;683103;247768;661200;197638;683086;683095;658520;683078;654677;683081;683080;200839;487419;407559;683068;34923;551116;683059;387824;683053;683054;419223;297715;683050;292907;293642;683035;683044;683042;683045;124122;683041;682501;61846;581216?site=superuser&key=lL1S1jr2m*DRwOvXMPp26g((&access_token=eIDDpZG(uvLciqdnktsQmw))
https://superuser.com/questions GET https://api.stackexchange.com/2.2/questions?pagesize=5&order=desc&sort=activity&site=meta.superuser
https://superuser.com/questions GET https://rawgit.com/shu8/SE-Answers_scripts/master/dupeClosedMigratedCSS.css
https://superuser.com/questions GET https://superuser.com/questions/1164534
https://superuser.com/questions GET https://superuser.com/questions/1164524
https://superuser.com/questions GET https://superuser.com/questions/1164522
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fsuperuser.com%2Fquestions%2F1164516%22&diagnostics=true
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fsuperuser.com%2Fquestions%2F1164496%22&diagnostics=true
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/qa-wb.js?v=9
(none) POST http://ocsp.digicert.com/
(none) GET https://qa.sockets.stackexchange.com/
https://superuser.com/questions GET https://superuser.com/questions/1164534/what-happens-if-i-sleep-windows-10-while-programs-are-running
https://superuser.com/questions GET https://superuser.com/questions/1164524/select-the-driver-to-be-installed-windows-7-home-premium-x86
https://superuser.com/questions GET https://superuser.com/questions/1164522/how-can-i-build-a-sound-display-to-show-the-alsa-master-volume-for-the-fn-keys
(none) POST http://ocsp.digicert.com/
(none) POST http://ocsp.digicert.com/
https://superuser.com/questions GET https://i.stack.imgur.com/Kmqx8.png
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/css/wb-include.css?22
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/theactualhats.js?v=8
https://superuser.com/questions GET https://metasmoke.erwaysoftware.com/xdom_storage.html
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/hat-78404333.js?v=8
https://superuser.com/questions GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=337631%3B607889%3BNaN&host=superuser.com&_=1483822563839
https://metasmoke.erwaysoftware.com/xdom_storage.html GET https://cdn.rawgit.com/ofirdagan/cross-domain-local-storage/d779a81a6383475a1bf88595a98b10a8bd5bb4ae/dist/scripts/xdLocalStoragePostMessageApi.min.js
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/hat-78278539.js?v=8
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=NaN&host=superuser.com&_=1483822104590
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file POST https://superuser.com/posts/1164582/editor-heartbeat/answer
https://0.docs.google.com/document/d/1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0/xdbcm.html?internal=false&gcp=%7B%22qlc%22%3Anull%2C%22elh%22%3Afalse%2C%22eoo%22%3Atrue%7D&rt=j&xpc=%7B%22cn%22%3A%22docs%22%2C%22tp%22%3A1%2C%22osh%22%3Atrue%2C%22ppu%22%3A%22https%3A%2F%2Fdocs.google.com%2Frobots.txt%22%2C%22lpu%22%3A%22https%3A%2F%2F0.docs.google.com%2Frobots.txt%22%7D GET https://0.docs.google.com/document/d/1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0/bind?id=1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0&sid=1cc64febbb900314&VER=8&token=AC4w5VilbfesZ4ZFdH5AoL0ZwlR4uNlZ2g%3A1483822100782&lsq=1482517927748&u=18029933486027320144&c=1&w=1&gsi=0&smv=2&RID=rpc&SID=EA75E4E583E22ADB&CI=1&AID=48&TYPE=xmlhttp&zx=r45bxqz5hyo4&t=1
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=NaN&host=superuser.com&_=1483822104591
https://chat.stackexchange.com/rooms/114/ask-a-super-user-moderator POST https://chat.stackexchange.com/events