Recording database restoration

DVBLink TVSource is discussed here!

Recording database restoration

Postby politby » Wed Jan 11, 2017 12:52 pm

As we all know (at least on Windows) the DVBlink server will reset the recording path to the default location anytime it discovers it has lost access to it. This could be for any number of reasons - a temporary network problem, a failed hard drive, etc.

When I restore the setting to the correct path and restart the server, a new empty dlrecorder.db is created and the previous one renamed to dlrecorder(1).db. After this the recording library is empty.

What then is the correct procedure to restore the library?

I have tried stopping the server, renaming dlrecorder(1).db back to dlrecorder.db and restarting followed by the repair command but that does nothing. What is that command supposed to do anyway?

What are the files stored in the backup subfolder good for?

This gives DVBlink pretty much zero resiliency against even small temporary problems. As I see it the current design has a few fundamental shortcomings:

  • It is possible to specify a network recording path but even a temporary network glitch will cause an empty recording database which at least I am incapable of restoring
  • No facility for separation of recording and long term storage of recordings
  • No facility for keeping metadata with the recordings

Oberon?
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345


Re: Recording database restoration

Postby politby » Tue Feb 07, 2017 10:26 am

Bump.

Why is DVBlogic not responding to this? No solution?
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby Videonisse » Wed Feb 08, 2017 8:41 am

+1 (has same questions)
Win 7 64-bit SP1, Intel i5-3570K 3,4GHz HD 4000 Graphics, 6GB, TBS 6984 S2 Quad,TBS 6280 T2 Dual
DVBLink 5.5.0 Connect & TVSource (mc_recorder), CD S Thor 0.8W,FTA Aastra 19.2E, S FTA T2, ACAMD 0.6.2
Clients iPhone 4S 5S 6S, iPad2 iOS 9,Win 7 PCs
Videonisse
 
Posts: 573
Joined: Mon Nov 21, 2011 10:29 pm
Location: Stocholm, sweden

Re: Recording database restoration

Postby the_man » Wed Feb 08, 2017 1:31 pm

    It is possible to specify a network recording path but even a temporary network glitch will cause an empty recording database which at least I am incapable of restoring
    No facility for separation of recording and long term storage of recordings
    No facility for keeping metadata with the recordings

Almost the same questions already were asked on the forum and there is an answers.
A short remark:
- network path is not a good option as you lose connection to it, or some other issue may happen. But DVBlink always tries to make a scheduled recording, hence it use/create a new rec database, which are reachable for sure....
There is no a metadata, as dvblink sever designed as a back-end for a dvblink clients as a front-endsm and they usually get all info from the same database.
Yes, it could be (and it is ) not suitable for all usage cases. But really it is not possible to cover all cases.

May be interesting, if you already have not seen it , one of the dvblink user have wrote a useful app for working with the dlrecorder.db
viewtopic.php?f=55&t=29127
NETGEAR RNDU2000; Synology DS112j, DS 212j, DS213+, DS 712+; Western Digital WD ex2; Asustor AS3, RPI2 ...
PCTV 460,461e,292e; TT 4400, 4650CI;DVBSKy S960; DD octopus net; Inverto sat2ip multibox; TBS DVB-C ...
the_man
 
Posts: 4696
Joined: Mon Dec 02, 2013 1:11 pm

Re: Recording database restoration

Postby politby » Wed Feb 08, 2017 4:30 pm

With all due respect, not particularly helpful.

I agree that missed recordings should be avoided at all cost so a fallback recording location is certainly needed. But that should not automatically result in a borked database. The database should be restored as soon as the location is available again and any recordings made to the fallback location merged into the user defined location.

The current way of reverting to the default location (which is quite likely to not have enough space given that it was changed in the first place), without any notification whatsoever, is not acceptable.

This keeps happening to me regardless of whether the recording location is local or remote. The repair function has never worked and neither has renaming the extra dlrecorder.db file.

The import tool sort of works but it requires the imported files to be in a different location which creates a lot of unnecessary copying back and forth.

PLEASE fix or create a workaround for this.

Here is a link to a current database file:

https://1drv.ms/u/s!Akz3h_eK2R70gqYRFhhbeKMueuXOpg

According to the library listing there are 16 recordings but the actual number is 290 on disk. Some of these were imported using the tool you linked to following an earlier database loss.

At some point in the past 10 days the database was reset even though there has been no downtime on the server where recordings are stored.

Can you take a look?
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby the_man » Wed Feb 08, 2017 5:47 pm

I need the current and the old dlrecorder.db file as well. (maybe the one from the C:\Users\Public\Documents\DVBLink\backup\ ) or some one like dlrecording(1).db ...
NETGEAR RNDU2000; Synology DS112j, DS 212j, DS213+, DS 712+; Western Digital WD ex2; Asustor AS3, RPI2 ...
PCTV 460,461e,292e; TT 4400, 4650CI;DVBSKy S960; DD octopus net; Inverto sat2ip multibox; TBS DVB-C ...
the_man
 
Posts: 4696
Joined: Mon Dec 02, 2013 1:11 pm

Re: Recording database restoration

Postby politby » Fri Feb 10, 2017 12:03 pm

https://1drv.ms/u/s!Akz3h_eK2R70gqYoKpaH4At_fWh-JQ

thiis is from my other system which is having identical problems
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby the_man » Fri Feb 10, 2017 12:35 pm

I've just checked and unfortunately, all databases are almost empty.
NETGEAR RNDU2000; Synology DS112j, DS 212j, DS213+, DS 712+; Western Digital WD ex2; Asustor AS3, RPI2 ...
PCTV 460,461e,292e; TT 4400, 4650CI;DVBSKy S960; DD octopus net; Inverto sat2ip multibox; TBS DVB-C ...
the_man
 
Posts: 4696
Joined: Mon Dec 02, 2013 1:11 pm

Re: Recording database restoration

Postby Videonisse » Fri Feb 10, 2017 11:55 pm

the_man wrote:There is no a metadata, as dvblink sever designed as a back-end for a dvblink clients as a front-endsm and they usually get all info from the same database.

I don't understand this answer. Of course there is meta data for the recordings and they are stored in dlrecorder.db, for example in the "completed_recording" table. If the link between the data entry in the sqlite database and the actual recording (ts file) is lost then it's not possible to recover. That's a known problem.

If the data at least was stored as a xml-file in same location as the recording, then in case of database problem, DVBLink could scan the recording directory and recover eventual lost data.

What does DVBLogic think about this?
Win 7 64-bit SP1, Intel i5-3570K 3,4GHz HD 4000 Graphics, 6GB, TBS 6984 S2 Quad,TBS 6280 T2 Dual
DVBLink 5.5.0 Connect & TVSource (mc_recorder), CD S Thor 0.8W,FTA Aastra 19.2E, S FTA T2, ACAMD 0.6.2
Clients iPhone 4S 5S 6S, iPad2 iOS 9,Win 7 PCs
Videonisse
 
Posts: 573
Joined: Mon Nov 21, 2011 10:29 pm
Location: Stocholm, sweden

Re: Recording database restoration

Postby politby » Wed Feb 15, 2017 5:48 pm

Videonisse wrote:
the_man wrote:There is no a metadata, as dvblink sever designed as a back-end for a dvblink clients as a front-endsm and they usually get all info from the same database.

I don't understand this answer. Of course there is meta data for the recordings and they are stored in dlrecorder.db, for example in the "completed_recording" table. If the link between the data entry in the sqlite database and the actual recording (ts file) is lost then it's not possible to recover. That's a known problem.

If the data at least was stored as a xml-file in same location as the recording, then in case of database problem, DVBLink could scan the recording directory and recover eventual lost data.

What does DVBLogic think about this?


+1

This is a zero resiliency design.

Let's say the recorder loses its connection with the drive used as recording location, for whatever reason. This causes it to revert to the default location and a new dlrecorder.db is created. Any recordings made from this point on will be in the default location and the new database file. Now let's assume the original location becomes available again. We now have recordings in 2 separate locations each with its own dlrecorder.db. In order to restore functionality this would have to be merged back into one database pointing to the original intended location. As far as I understand there is no way to do this.

The same thing would be the case if the default location is the one in use normally but is changed temporarily because of a disk problem or similar.

This is pretty much a fundamental design flaw making recording an iffy proposition. IMO fixing this should have priority over almost anything else.
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby peches » Wed Feb 15, 2017 6:25 pm

+1

I want something similar in thread http://forum.dvblogic.com/viewtopic.php?f=70&t=31066 but I was brushed off. Shame.

peches
peches
 
Posts: 23
Joined: Mon Sep 14, 2009 7:27 pm

Re: Recording database restoration

Postby politby » Fri Feb 17, 2017 12:27 pm

I don't get it. Why is DVBlogic so reluctant to acknowledge that we have a point? No one in his right mind could claim that this is a smart design.

Even if a re-design of the database model is too much of a bitter pill to swallow, at least add a proper database backup function that preserves the links between metadata and actual recording files, with a utility to rebuild the database.

Heck, I'd write it myself if I could code worth a damn 8)
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby politby » Sat Feb 18, 2017 11:14 am

Okay, I guess I'll continue talking to myself until someone from DVBlogic responds with a plan :(

I am maintaining a DVBlink system for a friend who makes a lot of recordings. Yesterday we had to migrate 1.3TB of recordings to a different drive array because of space constraints.

Impossible without losing the database - or have I got this wrong? How do I do it?
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Re: Recording database restoration

Postby the_man » Tue Feb 21, 2017 11:44 am

politby wrote:Okay, I guess I'll continue talking to myself until someone from DVBlogic responds with a plan :(

I am maintaining a DVBlink system for a friend who makes a lot of recordings. Yesterday we had to migrate 1.3TB of recordings to a different drive array because of space constraints.

Impossible without losing the database - or have I got this wrong? How do I do it?

You may copy all recording to the same path as it was with the previous HDD and copy(replace) the dlrecorder.db file at the DVBlink/database/ or C:\Users\Public\Documents\DVBLink\ (depends on where the dvblink server is running)
All this movements should be done with disabled dvblink server.
NETGEAR RNDU2000; Synology DS112j, DS 212j, DS213+, DS 712+; Western Digital WD ex2; Asustor AS3, RPI2 ...
PCTV 460,461e,292e; TT 4400, 4650CI;DVBSKy S960; DD octopus net; Inverto sat2ip multibox; TBS DVB-C ...
the_man
 
Posts: 4696
Joined: Mon Dec 02, 2013 1:11 pm

Re: Recording database restoration

Postby politby » Tue Feb 21, 2017 1:09 pm

the_man wrote:
politby wrote:Okay, I guess I'll continue talking to myself until someone from DVBlogic responds with a plan :(

I am maintaining a DVBlink system for a friend who makes a lot of recordings. Yesterday we had to migrate 1.3TB of recordings to a different drive array because of space constraints.

Impossible without losing the database - or have I got this wrong? How do I do it?

You may copy all recording to the same path as it was with the previous HDD and copy(replace) the dlrecorder.db file at the DVBlink/database/ or C:\Users\Public\Documents\DVBLink\ (depends on where the dvblink server is running)
All this movements should be done with disabled dvblink server.


Understand, but I said migration i.e permanently moving the recordings to a different drive. Not an unlikely scenario.
politby
 
Posts: 127
Joined: Tue May 05, 2009 1:29 pm
Location: N59.45817 E18.39345

Next

Return to DVBLink TVSource

Who is online

Users browsing this forum: No registered users and 9 guests