Numpy: basics to advanced

---

Numpy: basics to advanced

---

NumPy is a powerful library for the Python programming language that is used for scientific computing and data analysis. Some of the key features of NumPy include:

1. N-dimensional arrays: NumPy provides the ndarray (n-dimensional array) object, which is a powerful and efficient way to store and manipulate large arrays of homogeneous data (e.g. integers, floats, etc.). Here is an example of creating a 1-dimensional array:

import numpy as np
# Creating a 1-dimensional array
arr = np.array([1, 2, 3, 4, 5])
print(arr) # prints [1 2 3 4 5]

2. Array operations: NumPy provides a wide range of mathematical and statistical functions that can be applied to arrays, such as addition, subtraction, multiplication, etc. Here is an example of performing element-wise addition on two arrays:

import numpy as np
# Creating two arrays
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
# Adding the arrays element-wise
c = a + b
print(c) # prints [5 7 9]

3. Broadcasting: NumPy allows operations to be performed on arrays of different shapes, automatically broadcasting the smaller array to match the shape of the larger array. Here is an example of broadcasting a scalar value to an array:

import numpy as np
# Creating an array
arr = np.array([1, 2, 3, 4, 5])
# Broadcasting a scalar value to the array
b = arr + 2
print(b) # prints [3 4 5 6 7]

4. Array slicing: NumPy allows you to extract parts of an array using slicing, which is similar to list slicing in Python. Here is an example of slicing a 1-dimensional array:

import numpy as np
# Creating a 1-dimensional array
arr = np.array([1, 2, 3, 4, 5])
# Slicing the array
sliced_arr = arr[1:3]
print(sliced_arr) # prints [2 3]

5. Fancy indexing: NumPy allows you to extract parts of an array using indices, which can be arrays of integers or Boolean values. Here is an example of fancy indexing:

import numpy as np
# Creating an array
arr = np.array([1, 2, 3, 4, 5])
# Creating an index array
index = np.array([True, False, True, False, True])
# Using the index array to select elements from the original array
fancy_indexed_arr = arr[index]
print(fancy_indexed_arr) # prints [1 3 5]

6. Linear algebra: NumPy provides a wide range of linear algebra functions, such as matrix multiplication, matrix inversion, eigenvalues, and singular value decomposition. Here is an example of matrix multiplication:

import numpy as np
# Creating two matrices
a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6], [7, 8]])
# Multiplying the matrices
c = np.dot(a, b)
print(c

---

7. Fourier Transform : 

 The numpy library in Python provides a function called numpy.fft.fft (Fast Fourier Transform) that can be used to perform the Fourier Transform on a given signal or data. The function takes a single argument, which is the signal or data to be transformed, and returns the transformed data in the form of complex numbers.

For example, to perform the Fourier Transform on a signal stored in a numpy array called signal, you can use the following code:

import numpy as np
transformed_data = np.fft.fft(signal)

The numpy.fft.fftfreq function can be used to generate the corresponding frequencies for the transformed data.

sample_rate = 1000
frequencies = np.fft.fftfreq(signal.size, 1/sample_rate)

You can also use numpy.fft.fft2 for 2-dimensional data and numpy.fft.ifft for Inverse Fourier Transform.

It’s important to note that the resulting transformed data is complex numbers, to get the magnitude of the transformed data you can use the numpy function np.abs().

---

8. Interoperability: 

To implement interoperability in numpy, you can use the various functions and methods provided by the library to convert numpy arrays to and from other data structures, and to read and write data to and from various file formats.

For example, to convert a numpy array to a list, you can use the tolist() method like this:

import numpy as np
arr = np.array([1, 2, 3, 4, 5])
list_from_arr = arr.tolist()
print(list_from_arr) # [1, 2, 3, 4, 5]

To convert a list to a numpy array, you can use the numpy.array() function:

my_list = [1, 2, 3, 4, 5]
arr_from_list = np.array(my_list)
print(arr_from_list) # array([1, 2, 3, 4, 5])

To save a numpy array to a CSV file, you can use the numpy.savetxt() function like this:

np.savetxt("my_array.csv", arr, delimiter=",")

To read a CSV file into a numpy array, you can use the numpy.genfromtxt() function:

loaded_array = np.genfromtxt("my_array.csv", delimiter=",")

You can also use the numpy.save() and numpy.load() functions to save and load arrays to and from binary files in .npy and .npz format respectively.

For interoperability with other libraries like pandas and scikit-learn, you can use the pandas.DataFrame() constructor to convert a numpy array to a pandas dataframe and vice versa using the pandas.DataFrame.values attribute. Similarly, numpy arrays can be used as input to scikit-learn's machine learning models.

In summary, numpy provides a wide range of functions and methods to convert its arrays to and from other data structures, read and write data to and from various file formats and API’s for other languages which allows for easy integration of numpy with other libraries and tools.

---

9. Memory-mapped files : 

A memory-mapped file is a way of accessing the data in a file on disk as if it were stored in memory. This can be useful when working with large arrays or other large data sets that do not fit into memory. Numpy provides support for memory-mapped files through the numpy.memmap class.

To create a memory-mapped file in numpy, you can use the numpy.memmap() function, specifying the file name and shape of the array:

mmapped_array = np.memmap("my_array.dat", dtype='float32', mode='w+', shape=(1000, 1000))

You can then use the memory-mapped array just like any other numpy array, and any changes made to the array will be automatically saved to the file on disk.

mmapped_array[0,0] = 3.14

You can also use the numpy.memmap() to load an existing memory-mapped file.

loaded_mmapped_array = np.memmap("my_array.dat", dtype='float32', mode='r', shape=(1000, 1000))

You can also use numpy.memmap() to load only a portion of the memory-mapped file into memory, by specifying the offset and shape of the desired portion of the array.

loaded_mmapped_array = np.memmap("my_array.dat", dtype='float32', mode='r', offset=1024, shape=(1000, 1000))

It’s important to note that, if you’re done with the memory mapped array and you want to close the file, you have to use the del keyword to delete the array, this will close the file and free the memory used by the array.

del mmapped_array

In summary, numpy’s numpy.memmap() class provides support for memory-mapped files, allowing you to work with large arrays or other large data sets that do not fit into memory, by treating a file on disk as if it were stored in memory. This can be useful when dealing with large datasets and can be useful for out of core computation and for shared memory parallelism.

---

10 . File I/O : 

Numpy provides support for reading and writing data to and from files through several functions such as numpy.save(), numpy.savez(), numpy.savetxt(), numpy.load(), numpy.loadtxt() and numpy.genfromtxt().

For example, to save a numpy array to a binary file in .npy format, you can use the numpy.save() function like this:

import numpy as np
arr = np.array([1, 2, 3, 4, 5])
np.save("my_array.npy", arr)

To load the array back from the file, you can use the numpy.load() function:

loaded_array = np.load("my_array.npy")

You can also use the numpy.savez() function to save multiple arrays into a single .npz file:

np.savez("my_arrays.npz", arr1=arr1, arr2=arr2)

and then load the arrays back from the file using numpy.load():

loaded_arrays = np.load("my_arrays.npz")
arr1 = loaded_arrays["arr1"]
arr2 = loaded_arrays["arr2"]

To save an array to a text file, you can use the numpy.savetxt() function, specifying the file name and the array:

np.savetxt("my_array.txt", arr)

To read a text file into a numpy array, you can use the numpy.loadtxt() function:

loaded_array = np.loadtxt("my_array.txt")

You can also use numpy.genfromtxt() to read and parse data from text files and return a numpy array.

loaded_array = np.genfromtxt("my_array.txt", delimiter=",")

It’s important to note that, when you save an array to a file using numpy.save() or numpy.savez(), the file contains a binary representation of the array, which takes up less space and can be read back more quickly than a text representation of the array. On the other hand, when you save an array using numpy.savetxt(), the file contains a text representation of the array, which takes up more space but can be read and understood by humans.

In summary, numpy provides a wide range of functions for reading and writing data to and from files, including binary files in .npy and .npz format, and text files. These functions can be used to easily save and load numpy arrays to and from files, making it easy to work with large data sets, share data with others, and perform out-of-core computation.

NumPy provides a large set of features for scientific computing and data analysis and it is widely used in the scientific Python ecosystem. It’s also the foundation for many other Python libraries like pandas and scikit-learn.