[ICCV 2023] VPD is a framework that leverages the high-level and low-level knowledge of a pre-trained text-to-image diffusion model to downstream visual perception tasks.
A tensorflow implementation of GraphGAN (Graph Representation Learning with Generative Adversarial Nets)
关键参数与规格
关键参数与规格
参数
值
语言
Python
类别
Model Development
子类别
Architecture
关键词
Physics
星标数
539
分叉数
85
贡献者
3
创建日期
2019-02-12
更新日期
2026-08-25
1 天星标数
0
7 天星标数
0
排名
17,442
上次排名
17,229
排名变化
-213
7 天星标排名
16,105
上次 7 天星标排名
16,112
7 天星标排名变化
7
总星标排名
18,260
上次总星标排名
18,236
总星标排名变化
-24
国家
USA
README 摘录
wavetorch Overview This python package provides recurrent neural network (RNN) modules for pytorch that compute time-domain solutions to the scalar wave equation. The code in this package is the basis for the results presented in our recent paper, where we demonstrate that recordings of spoken vowels can be classified as their waveforms propagate through a trained inhomogeneous material distribution. This package not only provides a numerical framework for solving the wave equation, but it also allows the gradient of
the solutions to be computed *automatically* via pytorch's automatic differentiation framework. This gradient computation is equivalent to the adjoint variable method (AVM) that has recently gained popularity for performing inverse design and optimization of photonic devices. For additional information and discussion see our paper: T. Fan, Wave physics as an analog recurrent neural network, Science Advances, vol. 2019 Components The machine learning examples in this package are designed around the task of vowel recognition, using the dataset of raw audio
recordings available from Prof James Hillenbrand's website. However, the core modules provided by this package, which are described below, may be applied to other learning or inverse design tasks involving time-series data. The package provides several individual modules, each subclassing . These modules can be combined to model the wave equation or (potentially) used as components to build other networks. - A wrapper which contains *one* or more modules, *zero* or more modules, and a single module. The module is a
convenient wrapper around the individual components and handles time-stepping the wave equation. If no probes are present, the output of is the scalar field distribution as a function of time. If probes are present, the output will be (by default) the probe values, but this output can be overridded to instead output the field distribution. Usage Propagating waves See study/propagate. py Optimization and inverse design of a lens See study/optimize_lens. py Vowel recognition To train the
model using the configuration specified by the file study/example. yml, issue the following command from the top-level directory of the repository: The configuration file, study/example. yml, is commented to provide information on how the vowel data is processed, how the physics of the problem is specified, and how the training process is configured. During training, the progress of the optimization will be printed to the screen. At the end of each epoch, the current state of the
model, along with a history of the model state and performance at all previous epochs and cross validation folds, is saved to a file. **WARNING:** depending on the batch size, the window length, and the sample rate for the vowel data (all of which are specified in the YAML configuration file) the gradient computation may require a significant amount of memory. It is recommended to start small with the batch size and work your way up gradually, depending on what
your machine can handle. Summary of vowel recognition results A summary of a trained model which was previously saved to
常见问题
记录了多少星标和分叉？
fancompute/wavetorch 有 539 个星标和 85 个分叉。
仓库排名是多少？
fancompute/wavetorch 排名第 17,442。
记录的语言和类别是什么？
fancompute/wavetorch 有 539 个星标，语言为 Python，类别为 Model Development。
This python package provides recurrent neural network (RNN) modules for pytorch that compute time-domain solutions to the scalar wave equation. The code in this package is the basis for the results presented in our recent paper, where we demonstrate that recordings of spoken vowels can be classified as their waveforms propagate through a trained inhomogeneous material distribution.
This package not only provides a numerical framework for solving the wave equation, but it also allows the gradient of the solutions to be computed automatically via pytorch's automatic differentiation framework. This gradient computation is equivalent to the adjoint variable method (AVM) that has recently gained popularity for performing inverse design and optimization of photonic devices.
For additional information and discussion see our paper:
The machine learning examples in this package are designed around the task of vowel recognition, using the dataset of raw audio recordings available from Prof James Hillenbrand's website. However, the core modules provided by this package, which are described below, may be applied to other learning or inverse design tasks involving time-series data.
The wavetorch package provides several individual modules, each subclassing torch.nn.Module. These modules can be combined to model the wave equation or (potentially) used as components to build other networks.
WaveRNN - A wrapper which contains one or more WaveSource modules, zero or more WaveProbe modules, and a single WaveCell module. The WaveRNN module is a convenient wrapper around the individual components and handles time-stepping the wave equation. If no probes are present, the output of WaveRNN is the scalar field distribution as a function of time. If probes are present, the output will be (by default) the probe values, but this output can be overridded to instead output the field distribution.
WaveGeometry - The children of this module implement the parameterization of the physical domain used by the WaveCell module. Although the geometry module subclasses torch.nn.Module, it has no forward() method and serves only to provide a parameterization of the material density to the WaveCell module. Subclassing torch.nn.Module was necessary in order to properly expose the trainable parameters to pytorch.
WaveSource - Implements a source for injecting waves into the scalar wave equation.
WaveProbe - Implements a probe for measuring wave amplitudes (or intensities) at points in the domain defined by a WaveGeometry.
To train the model using the configuration specified by the file study/example.yml, issue the following command from the top-level directory of the repository:
python ./study/vowel_train.py ./study/example.yml
The configuration file, study/example.yml, is commented to provide information on how the vowel data is processed, how the physics of the problem is specified, and how the training process is configured.
During training, the progress of the optimization will be printed to the screen. At the end of each epoch, the current state of the model, along with a history of the model state and performance at all previous epochs and cross validation folds, is saved to a file.
WARNING: depending on the batch size, the window length, and the sample rate for the vowel data (all of which are specified in the YAML configuration file) the gradient computation may require a significant amount of memory. It is recommended to start small with the batch size and work your way up gradually, depending on what your machine can handle.
Summary of vowel recognition results
A summary of a trained model which was previously saved to disk can be generated like so:
python ./study/vowel_summary.py
Display field snapshots during vowel recognition
Snapshots of the scalar field distribution for randomly selected vowels samples can be generated like so:
Display short-time Fourier transform (STFT) of vowel waveforms
A matrix of short time Fourier transforms of the received signal, where the row corresponds to an input vowel and the column corresponds to a particular probe (matching the confusion matrix distribution) can be generated like so:
