Data recovery

From this article, you will learn how to recover data from DVR and NVR video recorders. We will explore the peculiarities of how they store video files, why such files can be lost, and how they can be recovered. In the process, well be using a IP network video recorder Hikvision DS-7104NI-Q1/4P.





Video surveillance systems are becoming the standard option for stores and offices, for monitoring the situation around your house, apartment, or in the street. Not so long ago, surveillance systems tended to be based on personal computers, but these days, the most widespread option is autonomous NVR and DVR devices which no longer need a PC nearby.

An accidental failure or a commonplace scenario with deleting a video file accidentally may have disastrous consequences if the file in question was very important. Recorders keep their videos in a special format so extracting the data is not so easy as it seems. You will need specialized software provided by the device manufacturer, and if the file is deleted a special program capable of finding the lost data and displaying it. Lets start by analyzing the reasons causing data loss for video recorders, and then find out how to recover data from the hard disk.

What causes DVR data loss

Todays market offers a wide range of devices with lots of various functions. They can save recorded vide files to the single hard disk, or several disks both individual drives or drives combined into a unified logical space. Various recorders can use unique algorithms to compress video streams, and premium models offer password protection and even encryption for all video files. With all of these options, if the internal hard disk ever fails or there is an unexpected system error, the task of recovering data into a regular video format seems quite a challenge.

Most of the time, video files are lost as a result of:

• Recorder hardware issues;
• Software errors;
• Removing the files accidentally by the user, or because of creating a faulty video recording;
• Formatting the internal memory;
• Physical damage or file system errors.

When the hard disk breaks down, there is no way to recover the lost data on your own, and youll have to find a data recovery specialist to deal with the problem.

If the recorder breaks down

If the video recorder is out of order, there is nothing to be recovered because all the data is still there, on the hard disk inside the device. However, if you connect the disk to the computer, dont expect to see anything; what is more, the Disk Management will suggest initializing it. For cases like this, what you need is a special software tool provided by the recorder manufacturer which can help you see the video files. But what if you dont have such software? Try finding another video recorder of the same model or use a data recovery tool.

Recorder specifications

This recorder, Hikvision DS-7104NI-Q1/4P, supports third-party cameras and lets you connect as many as four IP cameras. An effective video compression system helps the device to use less space for storing videos.



In the back panel, there are two video output channels HDMI and VGA, a 100 Mbit Ethernet port, two USB ports, and 4 independent Power over Ethernet (PoE) ports, and of course, a power connector.



Inside, there is a slot for one hard disk and a connector supporting disks up to 6 TB.

Backup settings

If you need to recover an old video, you should remember that any video recorder has to work with a limited memory capacity, regardless of the model. Depending on the quality of the picture and the number of cameras, it can record video files for a certain period of time, and then it will overwrite older data with new files. If a new record cycle has already begun, its unlikely that older videos can be recovered.

In general, before doing anything to the information which you think is important formatting, deleting, partitioning and so on always remember to back up your stuff. Most recorder models let you configure the backup process.



In this menu, you can schedule backups, specify what cameras should be connected, select the video format and change other settings.



In case you forgot to back up important files, you can still find a solution and use a special data recovery tool, Hetman Partition Recovery.

Recovering erased video files

Start by taking the recorder apart, disconnecting the hard disk, and connecting it to a Windows computer.

Many video recorders have file systems which are not typically used with computer operating systems. That is why you cant download or save any data from the disk you have connected to the computer. To have the video files recovered you will need either specialized equipment or dedicated software provided by the vendor of your surveillance system. This way or another, it suggests more expenses, and sometimes it just means there is no way to recover anything on your own.

At the same time, most video recorders use compatible files systems and this is also the case with the device we are testing. Therefore, the process of saving information is very simple just connect the hard disk to the computer and save the data.

Take out the screws and open the cover.



Take out the hard disk, disconnect it from the recorder, and connect the disk to the computer.



When connected, the disk is not displayed in the Windows Explorer, and the operating system suggests to initialize it but its not necessary now, so just hit Cancel.



As we know, initialization will erase all the data which is still on the disk, and any recovery operations you conduct later can hardly be successful.

Hetman Partition Recovery can recognize the disk, and it detect its file system as hikvision.



Right-click on the disk and open it.



Select Fast scan and wait until the process is complete.



If the Fast scan result is unsatisfactory, try a Full analysis instead.



The program has found the remaining data like a charm.



Most video recorders assign same-type names to their files, and such names are not too informative for us, humans. To find a specific video, use the preview function.



Select the video files you need to extract from the disk and click Recovery,



Specify the directory where youd like to save the information and click Recovery.



When the process is over, you will find the recovered files in the folder you have selected.

Recorders with RAID support

Also, there are premium recorders or full-featured professional models that save recorded videos to a RAID array. Depending on the RAID type, recorders can gain an extra bonus in:

Significantly higher read/write speed,

Ample storage space,

Improved resilience,

Automatic backup features and other things.



As video surveillance systems typically work round the clock, they are overwriting data on the disk all the time. With such intense workloads, 24/7 operation results in physical wear of the hard disk. That is why even in premium systems (for example, server-based ones) hard disks tend to be rather short-lived.

In order to secure your system against data loss issues, you should consider the hard disk excessive wear factor and make sure that HDDs you are using are replaced every two or three years. Another method is using recorders with RAID storage systems. It can extend the lifespan of the hard disks by a few years due to employing higher number of drives. In addition, youll be secured against loss of data even if one or two drives fail at the same time.



It is recommended to build an array on the basis of a high-quality RAID controller. With the help of standard controller software, youll be able to configure and maintain the array on your own. Even if one of the drives fails, this software will help you identify the faulty disk, find out what happened and decide if it needs to be replaced.

If you want to recover information from a RAID array, use Hetman RAID Recovery. It will identify its properties automatically and rebuild the damaged RAID array, so that you can copy the important data to a safer location.



See the full article with all additional video tutorials.

Would you like to know how to use the LVM technology in Linux? How to create an LVM volume, how to configure and mount it in your operating system, how to add and remove disks, and how to recover the information you have accidentally deleted? In todays article, you will find all of that and even more.

What is LVM?

LVM (Logical Volume Manager) is a standard disk management feature available in every version of Linux. The volume manager provides a new level of interaction between your operating system and disks or volumes which this system is using. In a conventional disk management pattern, Linux looks for available drives first, and then checks what volumes are available on these devices.

With the help of LVM, disks or volumes can be united into a single space. If the disk space is treated like that, the operating system will see no difference between them and the volume manager will show the operating system only the necessary volume groups or physical disks.



In fact, LVM has several considerable advantages to offer:

It can create a logical volume based on any number of physical disks which will be represented in the operating system as a single disk space.
The number of disks included and the size of such volume can be modified without interrupting your work.
Also, the LVM technology lets you do many things on the fly, like copying certain volumes, or configuring the mirroring feature the way it works in RAID 1 systems.

How to add LVM to the operating system?

If you are going to use LVM options for the first time, you have to install it. Here is the command you need for installation: sudo apt install lvm2



After that, type the administrators password, and click Yes to confirm your intention.



Some Linux versions have this feature installed by default.

This can be checked easily, by typing the command lvm in the terminal window.



Now you can move on to dealing with disks.

How to format a physical disk

Here we are, with three empty hard disks 320 GB each, which are unformatted and without any partitions.



The easiest way to format a disk is by using a disk management utility. Click on a disk open advanced settings then format it.



Specify the name and file system Next and click Format again to confirm your action



Now youll need the password to your administrator account.



Now that this disk is formatted, repeat the operation for every other disk which is not.

Also, a disk can be formatted with the terminal, using the command fdisk.

Sign in with a superuser account sudo i type the password.



Now run the following command: fdisk /dev/sdb where sdb is the unformatted disk.



Before you continue, make sure you have selected the right disk, because this operation erases all data from there.

Type n new volume, p primary, 1 first partition, then press Enter twice.



Now lets get the volume ready to be used with LVM. Type t to change the volume type, and then 8e to assign LVM type.



Check the volume properties by pressing p you can see everything is OK, a new formatted volume with the name sdb1 appeared; press w to record the changes. Similarly, format all the disks which are not formatted yet.



Now create an LVM volume on the partition you have just created, by entering the command: pvcreate /dev/sdb1.



Repeat the command for every disk: pvcreate /dev/sdс1, pvcreate /dev/sdd1.

Create a volume group

Combine the three disks you have just prepared into an LVM volume. Heres the command to use:

vgcreate vg1 /dev/sdb1 dev/sdc1 dev/sdd1

where vg1 is the name of the new group. Actually, you can use any name you prefer, but its recommended to put the vg letter combination before the name so that you know it is a volume group.

Create a logical volume to use LVM

When the disks are combined into the group, you need to create a new logical volume, so that you can use this group. For this purpose, type the command:

lvcreate -L 10G -n lv1 vg1

Where the -L stands for the size of the logical volume it is 10 GB in this case, and the -n assigns a name to the volume, and the vg1 indicates from which volume the space should be taken.



If there is any data on the disk, the operating system will warn you about it: type yes to confirm your decision to erase the information.

How to format and mount a logical volume?

The last step you still have to take is to format the volume in the disk management utility and mount it. Click on it to open advanced settings and then format it; specify the name and file system Next and click Format again.



Type the root (superuser) account password, and click here to start mounting. Now you can use the new volume.



If you need to format it with the terminal, type the following command:

mkfs -t ext4 /dev/vg1/lv1

Specify the file system as Ext4, the group name, and the volume name.



For mounting, type the following: mkdir /mnt/lv1, and then mount -t ext4 /dev/vg1/lv1 /mnt/lv1.



The volume is mounted.

How to change the volume size?

One of the advantages offered by logical volumes is the opportunity to add a new hard disk on the fly and expand the volume group. And on the contrary, if there is a hard disk in the volume group which is not in use, you can remove it.

There are three main tools to make physical volumes, volume groups and logical volumes larger or smaller.

• Resize this command shrinks or extends physical and logical volumes, but cant handle volume groups;
• Extend use it to increase logical volumes or volume groups;
• Reduce it can make logical volumes or volume groups smaller.

How to add a new hard disk to a volume group

The first step when you want to add a hard disk to a group is to format it. Scroll up to find a detailed explanation of this step.

After that, run the following command: vgextend vg1 /dev/sde1



where vg1 is the group name, and sde1 is the new (formatted) disk.

How to display detailed information about LVM

To display detailed information on the composition of an LVM group, run the command pvdisplay.



It displays the path to a physical disk, the name of the virtual group where it belongs, the information on the free and used disk space, disk ID and many other things.

How to remove a logical volume

To remove a logical volume, make sure that is disconnected (unmounted), then run the lvremove command, and the volume will be removed.

This command will help you to remove a volume group, but before that, you need to unmount the logical volume, so use another command first: umount /mnt/lv1



To proceed with removal, type this command: lvremove /dev/vg1/lv1, and type Yes to confirm this operation.



When the removal operation is successful, youll see this message.

Now the group can be removed, with this command: sudo vgremove vg1



Now it is time to remove all the disks that used to constitute the group; this command will do it: sudo pvremove /dev/sdb1 /dev/sdc1 /dev/sdd1



All the formatting of the physical disk is removed, and now the disk is back to its original condition.

How to recover deleted data from an LVM volume

If you accidentally removed important information from an LVM logical volume, you can still restore it with the help of Hetman Partition Recovery.

This utility supports LVM partitions and guarantees data recovery from there. It can handle most popular file systems used in Windows, Linux, Unix, and MacOS. It can ignore errors of the logical structure, scan the hard disk and restore the files you are looking for.

At the moment, there is no version of Hetman Partition Recovery for Linux. However, you can still run it with the help of a virtual machine based on Windows, install it as the secondary operating system on your PC, or connect the hard disk to another computer.

Start the program: after initialization, it recognized the logical volume without effort and now it displays all the data available on this volume.



Start with running Fast scan right-click on the disk, Open, Fast scan and when the scan is over, the program will display the results in the right side of the window. Removed elements are marked with a red cross.



If the Fast scan doesnt find the data you need, then use Full analysis.



Select the files to be restored, click Recovery, choose where to save them, then click Recovery again.



When the process is over, you will find all the files in the folder you have selected.

Conclusion

In this article, we did our best to explain how to use Logical Volume Management an integrated feature of Linux. We have reviewed all important stages: creating and configuring an LVM partition, adding and mounting new hard disks, shrinking and removing volumes. The LVM feature has been developed with servers in mind, but now you can enjoy all of its benefits on a desktop PC at home.

See the full article with all additional video tutorials.

Введение

Намедни попал в нашу лабораторию довольно любопытный заказ. Профессиональный диктофон ZOOM H6 Handy Recorder. Устройство и носитель в рабочем состоянии, проблема носитель, на котором находились данные (карта памяти SDHC на 32 Гбайт) был отформатирован в фотоаппарате. Вопрос о том, каким образом флешка из диктофона попала в фотоаппарат, оставим для истории нам ответ на него не интересен. Интересно восстановить записанные звуковые данные.

Форматирование разделов FAT32 это всегда потеря файловых записей; процесс сопровождается их заменой чистыми. С одной стороны, это удобно, так как в структурах FAT32 не остается никаких рудиментов файловых записей. С другой это крайне губительно для данных, так как в случае фрагментации файлов и необходимости их восстановить, простое RAW-сканирование становится недостаточным, на выходе получаются поврежденные или неработающие файлы.

Не исключение и наш заказ. Попытки отсканировать карту памяти имеющимся ПО (Data Extractor, R-Studio, R.Explorer) структуру папок обнаружить не смогли. При черновом восстановлении аудиопотока (wav-файлы) получаются работающие файлы, но с перемешанным содержимым то есть продолжительность звуковой дорожки верная, файл проигрывается, но звук идет кусками то из одной дорожки, то из другой.

Владелец устройства сообщил, что запись велась на два микрофона в формате интервью, то есть аудиопоток должен быть идентичным, но громкость голосов в каждой записи разной.


Поврежденный аудиофайл. Явно заметно, что области громкого звука и тихого звука имеют примерно одинаковую продолжительность и резкие, обрывающиеся, границы


Тот же аудиофайл после исправления.

ТТХ устройства

Начинаем, как водится, со знакомства с устройством. Идем на сайт производителя, и вдумчиво читаем, что там про него пишут. Особенно нам интересны поддерживаемые форматы файлов и количество независимых микрофонов. Устройство поддерживает подключение до 4 микрофонов и, соответственно, одновременную запись с них. Максимально может записываться одновременно 8 дорожек (оно и понятно: стерео с 4 микрофонов). С форматами файлов все тоже понятно. Ассортимент не велик, форматы широко распространенные, никаких своих форматов нет. И это хорошо.


Форматы аудиофайлов, поддерживаемые рекордером

Вся нужная информация найдена. Начинаем работать. Для восстановления информации нам нужно ответить последовательно на следующие вопросы: какой формат записи использовался в нашем случае, как расположены данные на карте памяти, и что надо сделать для того, чтобы файлы заработали правильно?

1. Какой использовался формат записи?

Ответ на этот вопрос находится в заголовке аудиофайла:



Тут все просто, в пояснениях не нуждается. Поехали дальше.

2. Как расположены данные на карте памяти?

Здесь и дальше используем WinHex.

Находим первый заголовок аудиопотока (берем пример заголовка из результатов предварительного RAW-восстановления). Выглядит он примерно так:



В каждом файле есть таймштамп:



Поиск тех аудиоданных, которые нам нужны, лучше вести не по заголовкам файлов (их могут оказаться, и оказалось в нашем случае, многие сотни), а по таймштампам.

Производим поиск от начала до первого таймштампа нужной нам аудиозаписи. Обозначаем начало сектора с таймштампом как начало блока. Теперь производим поиск последнего таймштампа нужной нам аудиозаписи (ищем, соответственно, с конца). Дальше нам нужно найти окончание данных, относящихся к нужной нам записи. Это просто: ищем следующий за последним таймштампом заголовок нового аудиопотока. На всякий случай нужно убедиться, что он не относится к тому, что мы ищем: смотрим в сектор, видим там другой таймштамп. Отлично. Последний байт перед найденным заголовком отмечаем как конец блока.

Теперь у нас имеется выделенная область, содержащая данные нужного нам аудиопотока:



Копируем эту область в новый файл. Так, во-первых, удобнее работать. А во-вторых, другие-то данные нам не нужны зачем нам их восстанавливать?

Дальше работаем с полученным файлом.

Из формата файла: WAV, 44k, mono. То есть при записи с двух микрофонов мы имеем дело с двумя звуковыми дорожками.

Разбиваем файл на фрагменты по количеству таймштампов. Получаем четное количество фрагментов со следующими параметрами:

Фрагмент 1 32 кбайт;
Фрагмент 2 кратно 32 кбайт;
Заголовки фрагментов 1 и 2 идентичны.

Фрагмент 3 32 кбайт;
Фрагмент 4 кратно 32 кбайт;
Заголовки фрагментов 3 и 4 идентичны.

И так далее, все имеющиеся аудиопотоки (восстановить требовалось 10).

После несложных вычислений:

1) Аудиопоток фрагментирован;
2) Размер фрагментов составляет 32 кбайт;
3) Фрагменты чередуются в порядке: четные относятся к первой дорожке, нечетные ко второй.


Структура аудиопотока, записанного с двух микрофонов рекордером ZOOM H6 Handy recorder

3. Пора собирать аудиофайлы

Очевидно, что после того, как мы узнали, каким образом ZOOM H6 handy recorder пишет данные, собрать их в нормально проигрываемые аудиофайлы не составило никакого труда и заняло несколько минут машинного времени:

1) Разбиваем весь выделенный аудиопоток на куски по 32 кбайт; мы использовали File Splitter & Cutter;
2) Четные куски отправляем в одну папку, нечетные в другую; использовался Total Commander, выделение по маске (кнопка "+" на цифровой клавиатуре) *.*1; *.*3; *.*5; *.*7; *.*9;
3) Переименовываем файлы в обеих папках так, чтобы их расширения представляли собой номера по порядку; снова Total Commander, групповое переименование и выбираем в расширении счетчик, а в параметрах счетчика не менее 5 цифр (так как фрагментов будет много);
4) Объединяем файлы по порядку в каждой из папок; опять же, удобен Total Commander;
5) Переименовываем результат в *.wav и наслаждаемся результатом =).

А что насчет других форматов файлов и другого количества микрофонов?

Конечно, это был интересный вопрос. И мы не преминули воспользоваться возможностью это проверить. Использовали три микрофона (больше просто не было в наличии), формат записи MP3, битрейт самый низкий и самый высокий из возможных.

Результат абсолютно идентичен предыдущему, с той лишь разницей, что в начале аудиопотока не два куска с заголовками, а три. Размер блока при записи те же 32 кбайт. Расположение кусков также по порядку:



Собрать файлы можно также, без напряжения, используя общеизвестный софт; единственное, что распределять куски по 32 кбайт придется по трем папкам. Делаем это с использованием скрипта для Total Commander. Ну а дальше групповое переименование, сборка и результат.

Заключение

Радует, когда производитель не заморачивается сложными собственными форматами данных, нагромождением алгоритмов их записи и тому подобной мутью, в разы увеличивающей необходимое на восстановление информации время.