Object detection has an increasing amount of attention in recent years due to its wide range of applications and recent technological breakthroughs. Deep learning is the state-of-art method to perform object detection. This task is under extensive investigation in both academics and real-world applications such as security monitoring, autonomous driving, transportation surveillance, drone scene analysis, robotic vision, etc., It is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images or videos. It not only provides the classes of the objects in an image but also localizes them in that particular image. The location is given in the form of bounding boxes or centroids. Instance segmentation may be defined as the technique that gives fine inference separately for each object by predicting labels for every pixel of that object in the input image. Each pixel is labeled according to the object class within which it is enclosed. We deal with Mask Region-Based Convolutional Neural Network (Mask R-CNN) to implement instance segmentation and detection of fire in a video or an image which can be used in real-world such as automatic fire extinguisher and alert systems. The training was done using Mask R-CNN for object detection with ResNet-101 backbone, with a 0.001 learning rate and 2 images per GPU. With this, the proposed framework can detect fire using Mask Region-Based Convolutional Neural Network and can send immediate alert to the user if fire is detected