让你在线看视频也能达到多线程下载的速度

推荐学习书目

› Learn Python the Hard Way

Python Sites

› PyPI - Python Package Index

› http://diveintopython.org/toc/index.html

› Pocoo

值得关注的项目

› PyPy

› Celery

› Jinja2

› Read the Docs

› gevent

› pyenv

› virtualenv

› Stackless Python

› Beautiful Soup

› 结巴中文分词

› Green Unicorn

› Sentry

› Shovel

› Pyflakes

› pytest

Python 编程

› pep8 Checker

Styles

› PEP 8

› Google Python Style Guide

› Code Style from The Hitchhiker's Guide

This topic created in 3224 days ago, the information mentioned may be changed or developed.

Video Funnel - 让你在线看视频也能达到多线程下载的速度

马上使用：

从 PyPI 安装：

$ pip(3) install --user video_funnel
# or
$ sudo pip(3) install video_funnel

启动 video_funnel 的服务器：

$ vf http://tulip.ink/test.mp4
======== Running on http://0.0.0.0:8080 ========
(Press CTRL+C to quit)

用 mpv 播放：

$ mpv http://localhost:8080

动机：

众所周知，百度网盘之类产品的视频在线播放非常模糊，下载吧又限速，于是我写了 aiodl 这个下载器，通过 EX-百度云盘获取的直链来“多线程”下载。可是每次都要下载完才能看又十分不爽，直接用 mpv 之类的播放器播放直链又因为限速的原因根本没法看，遂有了本项目。

实现思路：

先将视频按照一定大小分块。块的大小根据视频的清晰度而异，以下载完一个块后视频可以播放为准。可通过命令行参数 --block-size/-b 来指定，默认为 8MB。
对于上一步中的一个块，再次分块——为区别改叫切片，启动多个协程来下载这些切片，以实现“多线程”提速的目的。块和切片大小一起决定了有多少个连接在同时下载。切片的大小通过 --piece-size/-p 来指定，默认为 1MB。
一个块中的切片全部下载完后，就可以将数据传给播放器了。当播放器播放这一块的时候，回到第 2 步下载下一块数据。为节省内存，设置了在内存中最多存在 2 个下载完而又没有传给播放器的块。

一些细节：

该如何把数据传给播放器呢？我最初的设想是通过标准输出，这样简单好写。但 stdio 是无法 seek 的，这就意味着你只能从视频的开头看起，无法快进 :P 如你所见，现在的解决方案是用 HTTP 协议与播放器传输数据。需要快进的时候播放器发送 HTTP Range 请求，video_funnel 将请求中的范围经过分块、切片后“多线程”下载。但这样就又带来了两个问题：
1. 需要播放器支持从 URL 播放。mplayer、mpv 之类的命令行播放器大多都支持，但一些 Windows 的播放器就不得而知了 :P 不过可以使用 HTML 的 video 标签在浏览器播放。
2. 怎么就没有处理 Range 请求的包啊，自己处理很麻烦的好吗～
由于下载的部分是用异步 IO 写的，与播放器交互的服务器部分就不能使用 Flask 之类阻塞的框架了，幸好 aiohttp 居然同时支持客户端和服务端。
说起来简单，实际写起来处处是坑啊 :(

参加 https://www.v2ex.com/t/405569

GitHub: https://github.com/cshuaimin/video-funnel

Supplement 1 · Jan 8, 2018

刚刚把输出调整了以下，用上了进度条，大家快来更新啊 ;-D

$ pip install video_funnel --upgrade

$ vf https://tulip.ink/test.mp4 &
* Listening at port 8080 ...

$ mpv http://localhost:8080 --no-ytdl --quiet
Playing: http://localhost:8080
Block #0:  15%|████▏                        | 600k/4.00M [00:02<03:30, 17.0kB/s]

播放器

切片

下载

多线

28 replies • 2019-10-12 11:26:20 +08:00

nazor

Jan 5, 2018 via iPhone

挺好的，要是能用 js 实现就更好了。

davy1995

Jan 5, 2018 via Android

可以可以，很实用啊感觉

mingyun

Jan 6, 2018

$ pip install video_funnel
Collecting video_funnel
Could not find a version that satisfies the requirement video_funnel (from ver
sions: )
No matching distribution found for video_funnel

CSM

Jan 6, 2018 via Android

@nazor 对啊，要是 js 实现的就可以直接在浏览器播放了

@mingyun 你是不是没有用官方源？可能是还没有同步过去

zingl

Jan 6, 2018

youtube 就是这个套路

swulling

Jan 6, 2018 via iPhone

@mingyun 应该是 pip 版本太老升级下

diggerdu

Jan 6, 2018 via iPad

mo 一波 dalao 的 aiodl 真不错

madfloyd

Jan 6, 2018 via Android

大有用处。。。搞个视频网站

netsail

Jan 6, 2018 via iPad

前排围观支持！！！

ryd994

Jan 6, 2018 via Android

aria2c -x10 --stream-piece-selector=inorder

CSM

Jan 6, 2018

@ryd994 唉你看我这轮子造的，早知道有 aira2 这等神器还自己折腾什么。。。

CSM

Jan 6, 2018

不过 aira2 貌似不能快进？容我加上”边看边下“的功能 :)

linap

Jan 6, 2018 via Android

mark.摩拜大佬

wzhndd2

Jan 6, 2018

mark.摩拜大佬

jko123

Jan 6, 2018 via Android

收藏多，回复少，一群大佬准备大干

honkew

Jan 6, 2018

这个叫做串流播放

gabrielsong

Jan 6, 2018 via Android

膜拜大佬…

CSM

Jan 6, 2018 via Android

@chen2016 我也很奇怪为什么这么多收藏😂

zqqian

Jan 6, 2018

mark.摩拜大佬

wcsjtu

Jan 7, 2018 via Android

TS 流怎么搞……

CSM

Jan 7, 2018 via Android

@wcsjtu m3u8 吗？那就没办法了……

cnaol

Jan 7, 2018

Command "python setup.py egg_info" failed with error code 1 in C:\Users\UrAir\AppData\Local\Temp\pip-build-4koh515g\video-funnel\

怎么破

CSM

Jan 7, 2018 via Android

@cnaol 具体错误是什么呢？需要 Python 版本为 3.6，是不是这个原因？

cnaol

Jan 7, 2018

@CSM
> PS F:\myproject> pip install --user video_funnel
Collecting video_funnel
Using cached video_funnel-0.0.3.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\UrAir\AppData\Local\Temp\pip-build-v519sou0\video-funnel\setup.py", line 29, in <module>
'vf = video_funnel.__main__:main'
File "c:\users\urair\appdata\local\programs\python\python36\lib\distutils\core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\dist.py", line 315, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\dist.py", line 361, in fetch_build_eggs
replace_conflicting=True,
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 850, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 1122, in best_match
return self.obtain(req, installer)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources\__init__.py", line 1134, in obtain
return installer(requirement)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\dist.py", line 429, in fetch_build_egg
return cmd.easy_install(req)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\command\easy_install.py", line 653, in easy_install
not self.always_copy, self.local_index
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 636, in fetch_distribution
dist = find(requirement)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 617, in find
dist.download_location = self.download(dist.location, tmpdir)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 566, in download
found = self._download_url(scheme.group(1), spec, tmpdir)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 805, in _download_url
return self._attempt_download(url, filename)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 811, in _attempt_download
headers = self._download_to(url, filename)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 710, in _download_to
fp = self.open_url(strip_fragment(url))
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 747, in open_url
return open_with_auth(url, self.opener)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 948, in _socket_timeout
return func(*args, **kwargs)
File "c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\package_index.py", line 1067, in open_with_auth
fp = opener(request)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 544, in _open
'_open', req)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File "c:\users\urair\appdata\local\programs\python\python36\lib\urllib\request.py", line 1321, in do_open
r = h.getresponse()
File "c:\users\urair\appdata\local\programs\python\python36\lib\http\client.py", line 1331, in getresponse
response.begin()
File "c:\users\urair\appdata\local\programs\python\python36\lib\http\client.py", line 297, in begin
version, status, reason = self._read_status()
File "c:\users\urair\appdata\local\programs\python\python36\lib\http\client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "c:\users\urair\appdata\local\programs\python\python36\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "c:\users\urair\appdata\local\programs\python\python36\lib\ssl.py", line 1009, in recv_into
return self.read(nbytes, buffer)
File "c:\users\urair\appdata\local\programs\python\python36\lib\ssl.py", line 871, in read
return self._sslobj.read(len, buffer)
File "c:\users\urair\appdata\local\programs\python\python36\lib\ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\UrAir\AppData\Local\Temp\pip-build-v519sou0\video-funnel\

CSM

Jan 7, 2018 via Android

@cnaol 看最后一行，socket.timeout: The read operation timed out 应该是网络原因，换个 pip 源或挂代理试试

learningman

Feb 25, 2019 via iPad

* Listening at port 8080 ...
Error handling request
Traceback (most recent call last):
File "/root/.local/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/root/.local/lib/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/root/.local/lib/python3.6/site-packages/video_funnel/__init__.py", line 85, in handler
del request.headers['Host']
TypeError: 'multidict._multidict.CIMultiDictProxy' object does not support item deletion
貌似库改了

vcheckzen

Mar 1, 2019 via Android

windows 下安装了找不到 vf 命令

gIrl1990

Oct 12, 2019

@ryd994 --stream-piece-selector= inorder 和 geom 有什么区别？