pystiche_papers.li_wand_2016

Title	Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis
Authors	Chuan Li and Michael Wand
Citation	[LW2016]
Reference implementation	Repository / Archive
Variant	Image optimization
Content loss	FeatureReconstructionLoss
Style loss	MRFLoss
Regularization	TotalVariationLoss

Behavioral changes

See also

Paper implementations

The following parts are affected:

• FeatureReconstructionLoss

• MRFLoss

• TotalVariationLoss

• target_transforms

Hyper parameters

See also

Paper implementations

content_loss()

Parameter	impl_params
	True	False
layer	"relu4_1"	"relu_4_2"
score_weight	2e1	1e0

target_transforms()

Parameter	impl_params
	True	False
num_scale_steps	0	3
scale_step_width	5e-2
num_rotate_steps	0	2
rotate_step_width	7.5

style_loss()

Parameter	impl_params
	True	False
layers	("relu3_1", "relu4_1")
layer_weights	"sum"
patch_size	3
stride	2	1
score_weight	1e-4	1e0

regularization()

Parameter	impl_params
	True	False
score_weight	1e-3

image_pyramid()

Parameter	impl_params
	True	False
max_edge_size	384
num_steps	100	200
num_levels	3	None 1
min_edge_size	64
edge	"long"

1

num_levels=None implies that the number of levels is automatically calculated depending on max_edge_size and min_edge_size. See pystiche.pyramid.OctaveImagePyramid for details.

API

pystiche_papers.li_wand_2016.images()

Return type

DownloadableImageCollection

class pystiche_papers.li_wand_2016.FeatureReconstructionLoss(encoder, impl_params=True, **feature_reconstruction_loss_kwargs)

Feature reconstruction loss from [LW2016].

Parameters

• encoder (Encoder) – Encoder used to encode the input.

• impl_params (bool) – If False, calculate the score with the squared error (SE) instead of the mean squared error (MSE).

• **feature_reconstruction_loss_kwargs – Additional parameters of a pystiche.loss.FeatureReconstructionLoss.

See also

pystiche.loss.FeatureReconstructionLoss

pystiche_papers.li_wand_2016.content_loss(impl_params=True, multi_layer_encoder=None, hyper_parameters=None)

Content loss from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained multi-layer encoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

pystiche_papers.li_wand_2016.FeatureReconstructionLoss

Return type

FeatureReconstructionLoss

class pystiche_papers.li_wand_2016.MRFLoss(encoder, patch_size, impl_params=True, **mrf_loss_kwargs)

MRF loss from [LW2016].

Parameters

• encoder (Encoder) – Encoder used to encode the input.

• patch_size (Union[int, Tuple[int, int]]) – Spatial size of the neural patches.

• impl_params (bool) – If True, normalize the gradient of the neural patches. If False, use a score correction factor of 1/2.

• **mrf_loss_kwargs – Additional parameters of a pystiche.loss.MRFLoss.

In contrast to pystiche.loss.MRFLoss, the score is calculated with the squared error (SE) instead of the mean squared error (MSE).

See also

• pystiche.loss.MRFLoss

• pystiche_papers.li_wand_2016.extract_normalized_patches2d()

pystiche_papers.li_wand_2016.style_loss(impl_params=True, multi_layer_encoder=None, hyper_parameters=None)

Style loss from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained multi-layer encoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche_papers.li_wand_2016.MRFLoss

Return type

MultiLayerEncodingLoss

class pystiche_papers.li_wand_2016.TotalVariationLoss(impl_params=True, **total_variation_loss_kwargs)

Total variation loss from [LW2016].

Parameters

• impl_params (bool) – If False, use a score correction factor of 1/2.

• **total_variation_loss_kwargs – Additional parameters of a pystiche.loss.TotalVariationLoss.

In contrast to pystiche.loss.TotalVariationLoss, the the score is calculated with the squared error (SE) instead of the mean squared error (MSE).

See also

• pystiche.loss.TotalVariationLoss

pystiche_papers.li_wand_2016.regularization(impl_params=True, hyper_parameters=None)

Regularization from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche_papers.li_wand_2016.TotalVariationLoss

Return type

TotalVariationLoss

pystiche_papers.li_wand_2016.perceptual_loss(impl_params=True, multi_layer_encoder=None, hyper_parameters=None)

Perceptual loss from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• multi_layer_encoder (Optional[MultiLayerEncoder]) – Pretrained multi-layer encoder. If omitted, multi_layer_encoder() is used.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche_papers.li_wand_2016.content_loss()

• pystiche_papers.li_wand_2016.style_loss()

• pystiche_papers.li_wand_2016.regularization()

Return type

PerceptualLoss

pystiche_papers.li_wand_2016.nst(content_image, style_image, impl_params=True, hyper_parameters=None, quiet=False)

NST from [LW2016].

Parameters

• content_image (Tensor) – Content image for the NST.

• style_image (Tensor) – Style image for the NST.

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• hyper_parameters (Optional[HyperParameters]) – If omitted, hyper_parameters() is used.

• quiet (bool) – If True, not information is logged during the optimization. Defaults to False.

Return type

Tensor

pystiche_papers.li_wand_2016.image_pyramid(impl_params=True, hyper_parameters=None, **image_pyramid_kwargs)

Image pyramid from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

• hyper_parameters (Optional[HyperParameters]) – If omitted, hyper_parameters() is used.

• image_pyramid_kwargs (Any) – Additional options. See ImagePyramid for details.

See also

• pystiche.pyramid.OctaveImagePyramid

Return type

OctaveImagePyramid

pystiche_papers.li_wand_2016.hyper_parameters(impl_params=True)

Hyper parameters from [LW2016].

Parameters

impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here.

Return type

HyperParameters

pystiche_papers.li_wand_2016.extract_normalized_patches2d(input, patch_size, stride)

Extract 2-dimensional patches from the input with normalized gradient.

If stride >= patch_size, this behaves just like pystiche.extract_patches2d(). Otherwise, the gradient of the input is normalized such that every value is divided by the number of patches it appears in.

Examples

>>> import torch
>>> import pystiche
>>> input = torch.ones(1, 1, 4, 4).requires_grad_(True)
>>> target = torch.zeros(1, 1, 4, 4).detach()
>>> # without normalized gradient
>>> input_patches = pystiche.extract_patches2d(
...    input, patch_size=2, stride=1
... )
>>> target_patches = pystiche.extract_patches2d(
...     target, patch_size=2, stride=1
... )
>>> loss = 0.5 * torch.sum((input_patches - target_patches) ** 2.0)
>>> loss.backward()
>>> input.grad
tensor([[[[1., 2., 2., 1.],
          [2., 4., 4., 2.],
          [2., 4., 4., 2.],
          [1., 2., 2., 1.]]]])

>>> import torch
>>> import pystiche
>>> import pystiche_papers.li_wand_2016 as paper
>>> input = torch.ones(1, 1, 4, 4).requires_grad_(True)
>>> target = torch.zeros(1, 1, 4, 4).detach()
>>> # with normalized gradient
>>> input_patches = paper.extract_normalized_patches2d(
...    input, patch_size=2, stride=1
... )
>>> target_patches = pystiche.extract_patches2d(
...     target, patch_size=2, stride=1
... )
>>> loss = 0.5 * torch.sum((input_patches - target_patches) ** 2.0)
>>> loss.backward()
>>> input.grad
tensor([[[[1., 1., 1., 1.],
          [1., 1., 1., 1.],
          [1., 1., 1., 1.],
          [1., 1., 1., 1.]]]])

Parameters

• input (Tensor) – Input tensor of shape \(B \times C \times H \times W\)

• patch_size (Union[int, Sequence[int]]) – Patch size

• stride (Union[int, Sequence[int]]) – Stride

Return type

Tensor

pystiche_papers.li_wand_2016.target_transforms(impl_params=True, hyper_parameters=None)

MRF target transformations from [LW2016].

Parameters

• impl_params (bool) – Switch the behavior and hyper-parameters between the reference implementation of the original authors and what is described in the paper. For details see here. In addition, if True, every transformation comprises a valid crop after the rotation to avoid blank regions. Furthermore, the image is rescaled instead of the motif, resulting in multiple image sizes.

• hyper_parameters (Optional[HyperParameters]) – Hyper parameters. If omitted, hyper_parameters() is used.

See also

• pystiche.loss.MRFLoss.scale_and_rotate_transforms()

Return type

Sequence[Module]

pystiche_papers.li_wand_2016.preprocessor()

Preprocessor from [LW2016].

Return type

CaffePreprocessing

pystiche_papers.li_wand_2016.postprocessor()

Postprocessor from [LW2016].

Return type

CaffePostprocessing

pystiche_papers.li_wand_2016.multi_layer_encoder()

Multi-layer encoder from [LW2016].

Return type

MultiLayerEncoder

pystiche_papers.li_wand_2016.optimizer(input_image)

Optimizer from [LW2016].

Parameters

input_image (Tensor) – Image to be optimized.

Return type

LBFGS