pystiche_papers.ulyanov_et_al_2016

Title	Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
Authors	Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Viktor S. Lempitsky
Citation	[ULVL2016] / [UVL2017]
Reference implementation	Repository / Archive
Variant	Model optimization
Content loss	FeatureReconstructionLoss
Style loss	GramLoss

Instance norm

The authors published an improved version [UVL2017] of their initial paper [ULVL2016] with only a single but significant change: they developed InstanceNorm2d and used it as drop-in replacement for BatchNorm2d layers. To account for this we provide instance_norm flag, which defaults to True.

The original authors also use the same repository for both implementations and only differentiate between them with the branches:

Paper	Branch
[ULVL2016]	texture_nets_v1
[UVL2017]	master

Behavioral changes

See also

Paper implementations

The following parts are affected:

• content_transform(),

• GramLoss(),

• ConvBlock(),

• level(),

• Transformer().

Hyper parameters

See also

Paper implementations

Although there are four possible combinations for impl_params and instance_norm only three are listed below. Since the hyper-parameters in both papers are equal, both combinations with ìmpl_params=False are equal and thus not reported.

content_loss()

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
layer	"relu4_2"
score_weight	1e0	6e-1	1e0

style_loss()

Parameter	impl_params
	True / True	True / False	False
layers	("relu1_1", "relu2_1", "relu3_1", "relu4_1")	("relu1_1", "relu2_1", "relu3_1", "relu4_1", "relu5_1")
layer_weights	"sum"
score_weight	1e0	1e3	1e0

content_transform()

Parameter	Value
edge_size	256

style_transform()

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
edge_size	256
edge	"long"
interpolation_mode	"bicubic"	"bilinear"

batch_sampler()

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
num_batches	2_000	300	200
batch_size	1	4	16

optimizer()

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
lr	1e-3	1e0

lr_scheduler()

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
lr_decay	0.8		0.7
delay	0		4

Miscellaneous

Parameter	impl_params / ìnstance_norm
	True / True	True / False	False
num_epochs	25	10

API

pystiche_papers.ulyanov_et_al_2016.content_transform(impl_params=True, instance_norm=True, hyper_parameters=None)

Content transform from [ULVL2016][UVL2017].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

Return type

Sequential

pystiche_papers.ulyanov_et_al_2016.style_transform(impl_params=True, instance_norm=True, hyper_parameters=None)

Style transform from [ULVL2016][UVL2017].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

Return type

Module

pystiche_papers.ulyanov_et_al_2016.images()

Images from [ULVL2016][UVL2017].

Return type

DownloadableImageCollection

pystiche_papers.ulyanov_et_al_2016.dataset(root, impl_params=True, instance_norm=True, transform=None, hyper_parameters=None)

Return type

Sized

pystiche_papers.ulyanov_et_al_2016.image_loader(dataset, impl_params=True, instance_norm=True, num_workers=0, pin_memory=True, hyper_parameters=None)

Return type

DataLoader

pystiche_papers.ulyanov_et_al_2016.content_loss(impl_params=True, instance_norm=True, multi_layer_encoder=None, hyper_parameters=None)

Content loss from [ULVL2016][UVL2017].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained MultiLayerEncoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche.loss.FeatureReconstructionLoss

Return type

FeatureReconstructionLoss

class pystiche_papers.ulyanov_et_al_2016.GramLoss(encoder, impl_params=True, **gram_op_kwargs)

Gram loss from [ULVL2016][UVL2017].

Parameters

• encoder (Encoder) – Encoder used to encode the input.

• impl_params (bool) – If True, normalize the score twice by the batch size.

• **gram_op_kwargs – Additional parameters of a pystiche.loss.GramLoss.

See also

• pystiche.loss.GramLoss

pystiche_papers.ulyanov_et_al_2016.style_loss(impl_params=True, instance_norm=True, multi_layer_encoder=None, hyper_parameters=None)

Style loss from [ULVL2016][UVL2017].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained MultiLayerEncoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche_papers.ulyanov_et_al_2016.GramLoss

Return type

MultiLayerEncodingLoss

pystiche_papers.ulyanov_et_al_2016.perceptual_loss(impl_params=True, instance_norm=True, multi_layer_encoder=None, hyper_parameters=None)

Perceptual loss from [ULVL2016][UVL2017].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained MultiLayerEncoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche_papers.ulyanov_et_al_2016.content_loss()

• pystiche_papers.ulyanov_et_al_2016.style_loss()

Return type

PerceptualLoss

pystiche_papers.ulyanov_et_al_2016.transformer(style=None, impl_params=True, instance_norm=True, levels=6)

Transformer from [ULVL2016][UVL2017].

Parameters

• style (Optional[str]) – Style the transformer was trained on. Can be one of styles given by images(). If omitted, the transformer is initialized with random weights.

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• levels (int) – Number of levels in the transformer. Defaults to 6.

Return type

Transformer

pystiche_papers.ulyanov_et_al_2016.training(content_image_loader, style, impl_params=True, instance_norm=True, hyper_parameters=None, quiet=False)

Training a transformer for the NST.

Parameters

• content_image_loader (DataLoader) – Content images used as input for the transformer.

• style (Union[str, Tensor]) – Style image on which the transformer should be trained. If the input is str, the style image is read from images().

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

• quiet (bool) – If True, not information is logged during the optimization. Defaults to False.

Return type

Module

pystiche_papers.ulyanov_et_al_2016.stylization(input_image, transformer, impl_params=True, instance_norm=False, hyper_parameters=None)

Transforms an input image into a stylised version using the transformer.

Parameters

• input_image (Tensor) – Image to be stylised.

• transformer (Union[Module, str]) – Pretrained transformer for style transfer or string to load a pretrained transformer.

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

Return type

Tensor

pystiche_papers.ulyanov_et_al_2016.hyper_parameters(impl_params=True, instance_norm=True)

Hyper parameters from [ULVL2016][UVL2017].

Return type

HyperParameters

pystiche_papers.ulyanov_et_al_2016.multi_layer_encoder()

Multi-layer encoder from [ULVL2016][UVL2017].

Return type

VGGMultiLayerEncoder

pystiche_papers.ulyanov_et_al_2016.preprocessor()

Return type

CaffePreprocessing

pystiche_papers.ulyanov_et_al_2016.postprocessor()

Return type

CaffePostprocessing

pystiche_papers.ulyanov_et_al_2016.optimizer(transformer, impl_params=True, instance_norm=True, hyper_parameters=None)

Optimizer from [ULVL2016][UVL2017].

Parameters

• transformer (Module) – Transformer to be optimized.

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

Return type

Adam

pystiche_papers.ulyanov_et_al_2016.lr_scheduler(optimizer, impl_params=True, instance_norm=True, hyper_parameters=None)

Learning rate scheduler from [ULVL2016][UVL2017].

Parameters

• optimizer (Optimizer) – Wrapped optimizer.

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• instance_norm (bool) – Switch the behavior and hyper-parameters between both publications of the original authors. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

Return type

ExponentialLR