PyTorch is an open source machine learning framework that accelerates
the path from research prototyping to production deployment. Official
website: https://pytorch.org/. We call the C++ library of PyTorch as
LibTorch, the same below.
To build FFmpeg with LibTorch, please take following steps as
reference:
1. download LibTorch C++ library in
https://pytorch.org/get-started/locally/,
please select C++/Java for language, and other options as your need.
Please download cxx11 ABI version:
(libtorch-cxx11-abi-shared-with-deps-*.zip).
2. unzip the file to your own dir, with command
unzip libtorch-shared-with-deps-latest.zip -d your_dir
3. export libtorch_root/libtorch/include and
libtorch_root/libtorch/include/torch/csrc/api/include to $PATH
export libtorch_root/libtorch/lib/ to $LD_LIBRARY_PATH
4. config FFmpeg with ../configure --enable-libtorch \
--extra-cflag=-I/libtorch_root/libtorch/include \
--extra-cflag=-I/libtorch_root/libtorch/include/torch/csrc/api/include \
--extra-ldflags=-L/libtorch_root/libtorch/lib/
5. make
To run FFmpeg DNN inference with LibTorch backend:
./ffmpeg -i input.jpg -vf \
dnn_processing=dnn_backend=torch:model=LibTorch_model.pt -y output.jpg
The LibTorch_model.pt can be generated by Python with torch.jit.script()
api. https://pytorch.org/tutorials/advanced/cpp_export.html. This is
pytorch official guide about how to convert and load torchscript model.
Please note, torch.jit.trace() is not recommanded, since it does
not support ambiguous input size.
Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
All versions of MSVC that support C11 (namely >= v19.27)
also support the restrict keyword, therefore av_restrict
is no longer necessary since 75697836b1.
Reviewed-by: Martin Storsjö <martin@martin.st>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
At present, consume_update evaluates timeline state on all links for
a multi-input filter. This can lead to the filter being incorrectly
en/dis-abled when evaluation on a frame on a secondary link leads to
a different result than the frame on the current main link next in
line for processing.
In some builds, the following object files could be left behind
after make clean:
./libavfilter/metal/utils.o
./libavfilter/metal/vf_yadif_videotoolbox.metallib.o
./libavcodec/x86/h26x/h2656dsp.o
./libavcodec/neon/mpegvideo.o
./ffbuild/bin2c_host.o
Fixes: http://trac.ffmpeg.org/ticket/10895
Signed-off-by: Martin Storsjö <martin@martin.st>
src/libavfilter/internal.h:255:45: note: passing argument to parameter 'filter' here
int ff_filter_config_links(AVFilterContext *filter);
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Previously to support dynamic reconfigurations of the matrix string (e.g. 0m),
the rdiv values would always be cleared to 0.f, causing the rdiv to be
recalculated based on the new filter. This however had the side effect of
always ignoring user specified rdiv values.
Instead float user_rdiv[0] is added to ConvolutionContext which will store the
user specified rdiv values. Then the original rdiv array will store either the
user_rdiv or the automatically calculated 1/sum.
This fixes trac ticket #10294, #10867.
Signed-off-by: Stone Chen <chen.stonechen@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
Avoids ugly casts when uninitializing.
(One could actually avoid allocating this separately if one
were willing to expose FFFramePool to those files including
link_internal.h.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Also make FFFilterGraph.sink_links a FilterLinkInternal**
because sink_links is used to access FilterLinkInternal
fields.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(These fields were in AVFilterGraph although AVFilterGraphInternal
existed for years.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
To do this, allocate AVFilterGraphInternal jointly with AVFilterGraph
and rename it to FFFilterGraph in the process (similarly to
AVStream/FFStream).
The AVFilterGraphInternal* will be removed on the next major version
bump.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit moves the generic-layer stuff (that is not used
by filters) to a new header of its own, similarly to
5e7b5b0090 for libavcodec.
thread.h and link_internal.h are merged into this header.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
To do this, allocate AVFilterInternal jointly with AVFilterContext
and rename it to FFFilterContext in the process (similarly to
AVStream/FFStream).
The AVFilterInternal* will be removed from AVFilterContext
on the next major bump.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
These logs use the wrong loglevel and are uninformative;
and it is of course highly unlikely that a buffer of 56B
can't be allocated.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This hack is used to limit the visibility of some AVFilterLink fields to
only certain files. Replace it with the same pattern that is used e.g.
in lavf AVStream/FFStream and avoid exposing these internal fields in a
public header completely.
Makes it robust against adding fields before it, which will be useful in
following commits.
Majority of the patch generated by the following Coccinelle script:
@@
typedef AVOption;
identifier arr_name;
initializer list il;
initializer list[8] il1;
expression tail;
@@
AVOption arr_name[] = { il, { il1,
- tail
+ .unit = tail
}, ... };
with some manual changes, as the script:
* has trouble with options defined inside macros
* sometimes does not handle options under an #else branch
* sometimes swallows whitespace
avfilter_insert_filter() already reports (also with AV_LOG_VERBOSE)
when a filter is auto-inserted.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Otherwise, passing an UNSPECIFIED frame to am MPEG-only filter graph
would trigger insertion of an unnecessary vf_scale filter, which would
perform a memcpy to convert between the two.
This is safe to do because unspecified YUV frames are already
universally assumed to be MPEG range, in particular by swscale.
Currently, this only affects untagged RGB/XYZ/Gray, which get forced to
their corresponding metadata before entering the filter graph. The main
justification for this change, however, is the planned ability to add
automatic promotion of unspecified yuv to mpeg range yuv.
Notably, this change will never allow accidentally cross-promoting
unspecified to jpeg or to a specific YUV matrix, since that is still
bound by the constraints of YUV range negotiation as set up by
query_formats.
When this filter overrides frame properties, the outgoing frames have
a different YUV colorspace than the incoming ones. This requires
signalling the new colorspace on the outlink, and in particular, making
sure it's *not* set to a common ref with the input - otherwise the point
of this filter would be destroyed.
Untouched fields will continue being passed through, so we don't need to
do anything there.
An oversight in my previous series. This omission slipped under the
radar because fftools/ffmpeg_filter.c did not use these options, instead
preferring to insert an explicit format filter.
Niklas Haas
Wenbin Chen
Timo Rothenpieler
Andreas Rheinhardt
Rajiv Harlalka
Gyan Doshi
Martin Storsjö
联盟少侠
Anton Khirnov
Haihao Xiang
James Almer
Andrew Sayers
Zhao Zhili
Fei Wang
Stone Chen
Michael Niedermayer