The work presented in this thesis aims to understand the use of tensor algebra for background and foreground separation in videos. Specifically, it tries to explore the advantages of tensor-based approaches over the vector-based ones. In vector-based approaches, video frames are vectorized and concatenated into columns of a matrix for foreground and background separation. Through vectorization, one cannot explore the multi-dimensional aspect of video frames. Recent research has shown that tensor algebra can be helpful in extracting useful information from a multi-dimensional perspective. In this thesis, we propose two new algorithms which use tensor algebra to solve for background and foreground separation. In the first part of the thesis, we develop a mini-batch extension to Online Tensor Robust Principal Component Analysis (OTRPCA). The proposed extension significantly reduces the computational time in comparison to OTRPCA. It is also shown that the accuracy levels of background separation are higher than OTRPCA for a decent mini-batch size. As the mini-batch size further increases, accuracy levels fall as the dictionary update is one-shot and non-iterative. In the second part of the thesis, online vector-based Grassmanian Robust Adaptive Subspace Algorithm (GRASTA) is extended to tensor domain. The proposed Multi-Linear GRASTA (MLG) is also an online algorithm, thus suitable for real-time applications. Unlike the vector-based implementation, MLG explores the multi-dimensional nature of the video frames and solves for the separation problem across every dimension. MLG can process multiple frames at a time making it faster than other vector and tensor based separation algorithms. Detailed results are discussed which show that the accuracy of separation with MLG is competitive with the state-of-the-art.