如何在使用文件输入时使用wget --wait选项?

Modified on: Sat, 30 Jun 2018 11:16:30 +0800

我有一个文本文件,其中包含我要下载的网址列表。我希望wget在下载下一个url之前等待。我正在使用

cat urls.txt | xargs -n1 -i wget --wait=30 {}

然而,网址之间没有等待。我可以使用wget的--wait选项吗?其他替代方案?

作者:arjan

最佳答案

这是因为每个网址都会调用wget。使用-i选项将URL列表提供给wget

$ wget -i urls.txt --wait=30

来自手册:

-i file
--input-file=file

Read URLs from a local or external file. If - is specified as file,
URLs are read from the standard input. (Use ./- to read from a file
literally named -.)  If this function is used, no URLs need be present
on the command line. If there are URLs both on the command line and in
an input file, those on the command lines will be the first ones to be
retrieved. If --force-html is not specified, then file should consist
of a series of URLs, one per line.

However, if you specify --force-html, the document will be regarded as
html. In that case you may have problems with relative links, which
you can solve either by adding "<base rel="nofollow" href="url">" to the documents or
by specifying --base=url on the command line.

If the file is an external one, the document will be automatically
treated as html if the Content-Type matches text/html. Furthermore,
the file's location will be implicitly used as base href if none was
specified.
作者:,user61786

相关问答

添加新评论