2023 Learn by Example: Pytorch Random related

PyTorch provides a variety of random functionalities through its torch.random and torch modules.

1. Setting the Seed : You can set a seed for the random number generator to ensure reproducibility in your experiments. This is useful for debugging and sharing code.

torch.manual_seed(42)

2. Generating Random Tensors : You can create a tensor with random numbers uniformly distributed between 0 and 1 using torch.rand.

x = torch.rand(5, 3)

3. Normal Distribution : Generate random numbers from a normal (Gaussian) distribution with torch.randn.

x = torch.randn(5, 3)

4. Random Integers : You can generate random integers in a given range with torch.randint.

x = torch.randint(0, 10, (5, 3))

5. Permutation and Shuffling : You can create a random permutation of integers or shuffle the order of a tensor using torch.randperm.

perm = torch.randperm(5)

x = torch.tensor([1, 2, 3, 4, 5])

shuffled_x = x[perm]

6. Sampling from Distributions : PyTorch provides a torch.distributions module that includes various probability distributions like Bernoulli, Poisson, etc. You can sample from these distributions.

from torch.distributions.normal import Normal

dist = Normal(0, 1)

sample = dist.sample((5,))

7. Random Sampling with Replacement : You can use torch.multinomial to sample values from a tensor with or without replacement.

weights = torch.tensor([0.1, 0.2, 0.7], dtype=torch.float)

samples = torch.multinomial(weights, 10, replacement=True)

8. Random Initialization of Weights : PyTorch's nn.init module provides functions to initialize weights of neural network layers using different random techniques.

from torch.nn import init

layer = torch.nn.Linear(5, 3)

init.xavier_uniform_(layer.weight)

9. Controlling Randomness on GPU : If you are using CUDA, you can also set the seed for the GPU to ensure reproducibility across GPU computations.

seed = 42

torch.cuda.manual_seed(seed)

torch.cuda.manual_seed_all(seed)  # Set seed for all GPUs if multiple are available

10. State of Random Number Generator : You can get and set the state of the random number generator using the following methods. This can be useful if you need to save and restore the state of the random number generator between different runs of your program.

# Get current state

rng_state = torch.get_rng_state()



# Later, or in another program, restore the state

torch.set_rng_state(rng_state)