LineUp - A Channel Line-Up Utility

DVBLink Remote API clients and tools

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Wed Oct 09, 2013 6:43 pm

EPG processing is optional in LineUp (a configuration option). You can maintain the channel line up from the ARC and assign EPG manually within TVSource.
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm


Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Sun Oct 13, 2013 2:40 pm

I made some changes to my DVB-T tuners and would have used the TVSource 'Merge Channels' feature to add the channels back - except it is no longer present in TVSource 4.6. I see others are missing this feature, so I created a guide on how to use LineUp to achieve the same result. PDF files are not allowed on this forum - I have added it to the LineUp share (see post #1)
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby chopperno5 » Sun Oct 13, 2013 4:02 pm

PRBUK,

Really been quite impressed with how well the utility works having installed it finally - I have been holding back since launch as I was hoping that EPG collector would have an all-in-one solution for ARC handling over the course of this year.

However, I, like many others on here I am sure, use Digiguide for my EPG. I can provide you with data where possible - although to my eye the data size is somewhere in the region of 65MB. At the moment I get a lovely fresh lineup, up-to-date with the ARC every time, but keep losing my EPG settings in TVsource which is killing the WAF..

If you are interested in giving this a go, I am sure that between us users we can rattle up a donated license for the digiguide addin and a years subs to digiguide? Maybe even a few beers as well..!

What do the rest of you guys think?
Server: WHS 2011/i3-2120/8GB/SSD boot/ 2x BGT3595 (28.2/19.2E) TVSource + Net Pack 4.6.0/Acamd 6.2.0.GBitNet- Clients: HTPC: W7/Core i5-3570S/8GB/AMD7850/SSD/GBitNet/BluRay. Study: W8.1/i5-3470/8GB/GTX570/SSD/GBitNet & Iphone 4S x2.
chopperno5
 
Posts: 29
Joined: Sat Feb 19, 2011 8:39 pm
Location: Germany

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Sun Oct 13, 2013 4:58 pm

chopperno5 wrote:.....but keep losing my EPG settings in TVsource which is killing the WAF..

We cannot have that!

I will have a look at Digiguide over the winter. In the interim, make sure that the Digiguide EPG source is NOT defined in the LCF as an EPG source and then set <EPG_Update_Other> to 0 in the LCF. If you assign Digiguide EPG data within TVSource with these settings, LineUp will not attempt to update the channel's EPG data.
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby chopperno5 » Sun Oct 13, 2013 8:37 pm

Thanks for the very prompt reply - I shall amend the LCF accordingly and await further updates...!
Server: WHS 2011/i3-2120/8GB/SSD boot/ 2x BGT3595 (28.2/19.2E) TVSource + Net Pack 4.6.0/Acamd 6.2.0.GBitNet- Clients: HTPC: W7/Core i5-3570S/8GB/AMD7850/SSD/GBitNet/BluRay. Study: W8.1/i5-3470/8GB/GTX570/SSD/GBitNet & Iphone 4S x2.
chopperno5
 
Posts: 29
Joined: Sat Feb 19, 2011 8:39 pm
Location: Germany

Re: AW: LineUp - A Channel Line-Up Utility

Postby Heiko123 » Wed Oct 16, 2013 11:32 pm

Sorry, I have a simple question.

Understand i right, that tool make copies from a config-tuner to all other tuners?
Or
Can I MERGE differents groups with the same channels?

Gesendet von meinem ARCHOS 101G9 mit Tapatalk 2
W7MC-Rechner: GA-MA785GT-UD3H; AMD Athlon X4 630; beQuiet 300W; 8GB RAM; 8x DVB-S (2x PCTV4000i;2x DigitalDevices-DVBS2); W7MCE-32bit; DVBLink-Source 4.5.3; DVBLink-Connect; DVBLink-Client
... and you can find me: http://www.mce-community.de/forum/
Heiko123
 
Posts: 141
Joined: Mon Nov 07, 2011 9:54 pm

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Thu Oct 17, 2013 9:41 am

LineUp will not create or change the Tuner Source files, it can only add existing channels from the tuner sources into your selected channels list.

LineUp is controlled by its configuration file (LCF). This contains details of every channel that you want selected within TVSource. Each tuner source must be defined and scanned as a stand alone source in TVSource (not a group) and you specify the tuner names to be processed within the LCF. LineUp then checks each Tuner Source and ensures that the correct channels are selected.

LineUp will only select a channel if it is present in the scanned tuner source, it does not copy or create entries in the tuner sources - hence if you have a mix of DVBS and DVBS2 tuners, S2 channels will only be selected from the S2 tuners.

The idea behind LineUp was to apply the 'ARC' file produced by the excellent EPG Collector and hence remove the manual workload of constantly moving channels.

My post on Sunday 13th October was to address a situation where you do not have access to an ARC file, but have the desired channels selected from one tuner and want to add the corresponding channels from other tuners. You can therefore 'capture' the selected channels from the 'master tuner' into the LCF from TVSource as 'manual channels' and then add the additional tuner sources to have their corresponding channels added.

One note of caution - TVSource manipulates tuner names which contain 'special characters' in the Source Name. I do not have a definition of these rules. LineUp will not have problems with names that contain alpha (A-Z), numeric (0-9), space or underscore but may have issues with 'national characters' and other punctuation. Please ensure your source names meet these constraints.

Hope this helps.
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby MCE_DVB » Thu Oct 17, 2013 1:51 pm

Hi PRBUK,

Firstly thank you for this excellent work and for kindly sharing it.

I have finally got around to implementing this on my system and have read through the quick start and referenced the full manual also.

I have done trial runs and all works great so now I want to ensure I have all the correct bits and pieces in play.

Specifically I wanted to ask:

• I understand that the “List_ARC utility” will show all channels in a neat list in ‘reports’ as a result of those in the ARC. I also understand that the ARC holds data of all ‘bouquets’. I also agree that I then need to add said ‘area’ and ‘region’ to the LCF.

    o Why are there 2x areas and regions?
    o How can I understand which areas / regions are correct for me to add to the LCF. I know where in the UK I am but cant see a relervant correlation to a bouquet? (I am sure I am likely missing something obvious here or having a 'brain fart')

• If I imagine I now had everything all set and now I notice a channel has moved, I would run the Run_Process_ARC_LCF_Report.cmd check changes, then run Run_Process_ARC_LCF_Replace.cmd. I would now run Run_ SyncToTVSource_Local_LCF_Replace.cmd. as such having the ARC up to date is vital and this is where EPG collector comes in. I have installed this, selected just one sat tuner, then selected output tab and ticked “create area /region channel file” and then added output path C:\ProgramData\DVBLogic\dvblink\xmltv\Sky_TVGuide.xml, on channels tab clicked scan, then did ‘file’ > ‘save as’ and saved the file in the default way. Via task scheduler I have it set to run as needed, this produces the two files in the C:\ProgramData\DVBLogic\dvblink\xmltv\ directory.

    o Should I point / change the path for the ARC in the LCF to the file created at path C:\ProgramData\DVBLogic\dvblink\xmltv\**File name** or should I be doing something else? Such as using another program to move and copy / over write the file from one place to another?
Thank you for any clarity / help you can provide.

Once I can grasp the above I intend then to simply modify the LCF with the parameter needed so it matches the line-up I specifically want and then I should be set.
MCE_DVB
 
Posts: 164
Joined: Tue Nov 01, 2011 2:29 pm

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Thu Oct 17, 2013 6:06 pm

It's fun being a detective! List_ARC was written to give you something to study as this is all undocumented!

In the case of Sky, my conclusion is that each Area consists of a 'base Region' and then geographic region specific variations. For example, Area 4101 is described as "HD England". Take a look at the BBC 1 channel names on channel 101 in the various Regions within this Area to determine which Region you are in (whilst ITV, Channel 4 and Channel 5 are called the same in all Regions, you will notice that there are different nid/tid/sid values to pick up the correct regional variations). There is then a large Region called 65535 which has the channels which are common to all regions. If you are in London (Region 1), you will want to specify two bouquets in the ARC: 4101/1 and 4101/65535.

You will also notice that in Region 65535 there are many channels with the same channel number. My guess is that these are part of the interactive/'red button' service. Process_ARC will see this as multiple instances of a 'new channel on an existing channel number' which will clutter up the log - hence you can use the 'block list' to exclude these from ARC processing.

An earlier request was to be able to 'extract' channels (assign a different channel number) from the list of duplicate channel numnbers, hence 'pre edits' were added to LineUp. Pre-Edits are processed before the block list and before the ARC is processed!

I hope this answers your first question

With respect to the second, channel moves/changes are a very frequent occurence! If a channel moves, you need to update your system pretty fast (or else you may get the dreaded Play Ready until you do!). Similarly you will want to make sure that your EPG is up to date to catch any schedule changes. This is where EPG COllector comes in.

I run an update job every 6 hours via the Windows task scheduler with multiple steps:
  • First I run EPG collector to create the EPG and ARC files.
  • Next copy the EPG XML file into the DVBLogic xmltv folder from where it will be periodically read.
  • Then run 'Process_ARC' in 'New' mode. This is configured to read the ARC file just produced by EPG Collector against the 'live' LCF and save the resulting modified LCF to the new filename (see the manual about 'New' mode).
  • This 'new' LCF is then used as the input to a 'SyncToTVSource' job running in 'Report' mode (which can be run while the DVBLink server is running). This will report the impact of the changes and therefore if a tuner scan is required to pick up the new channels before running the update for real.

I have a second job which runs once per week (or on demand if required) which runs 'Process_ARC' in 'Replace' mode (and hence saves the changes to the live LCF), then stops DVBLink Server, takes a backup copy of the TVSource config files and the LCF and runs SyncToTVSource in 'Replace' mode - this therefore applies the changes to the live system.

So to your second question, your understanding is correct. EPG COllector overwrites the ARC file everytime it runs. You need to configure your LineUp jobs to point to this ARC file (and run EPG Collector regularly to ensure it is current) and place the EPG COlllector EPG XML file in the TVSource xmltv directory.

As you experiment, it should become clearer. Good luck - let us know how you get on!
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby MCE_DVB » Fri Oct 18, 2013 11:21 am

Hi PRBUK,

Thank you for the full reply – had to read it a few time but yes – I am with you.

So the reason for 2 regions / areas is one is generic to all (4101/65535) in the case of skyHD England. And one is regionally specific = 4101/1 (London) so I would find the one for me which would be 4101/? Where ‘?’ I will find from studying the ARC. I am with you – great stuff.

Since the generic region/area (4101/65535) has multiple channel number issues could I simply use ‘<exclude>1’ instead of the ‘block list’? Would they amount to the same thing?

•Next copy the EPG XML file into the DVBLogic xmltv folder from where it will be periodically read.

I do not need to move my EPG .xml file as this already by setting in EPG Collector outputs to the xmltv directory in DVBLink along with the ARC file created. So for me there is no file moving needed as you had described – if this causes a problem let me know. I have just pointed the LCF to the ARC in the DVBLink XMLTV directory instead of the D:Lineup/EPG directory.

I can see how your full automation must work well and I in the long run will do the same however for now I will take pigeon steps to get to grips with it all.

This really is a great set up and should you ever create a “donate” option somewhere I would be keen to buy you a beer in thanks for your work here.

I will keep you / all posted on progress! 
-LightsOn
MCE_DVB
 
Posts: 164
Joined: Tue Nov 01, 2011 2:29 pm

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Fri Oct 18, 2013 3:05 pm

Nearly there!

The <Exclude> tag is present on every channel in the LCF and determines whether SyncToTVSource adds the channel in TVSource or not. A typical full sky line-up consists of around 400 channels. If you want to see 100 of those in TVSource/Media Center etc, you can set <exclude>1 on the other 300. The next time SyncToTVSource runs, they will be removed from TVSource, but are still present in the LCF should you change your mind (and will still be maintained by Process_ARC).

As you noted, a full ARC file consists of tens of thousands of channel entries. Process_ARC has three 'filter' mechanisms to filter out the channels you are interested in.

The first extracts the channels in your specified bouquets. It will only allow one channel to exist on each channel number, hence if the resulting Bouquet contains multiple channels on the same channel number, it will process each entry in sequence and update the channel number with the 'latest' information. If you therefore process 50 channels with channel number 65535, the resulting LCF will contain only one channel 65535 and this will contain the 'physicals' from the last entry that was processed. The Process_ARC log will also show 50 instances of 'New channel on existing channel number' (try it in report mode!). The Block List is therefore the third filter applied to the ARC to prevent these unwanted channels being processed. The second filter was added in version 1304 in response to the request from 'elyl', who wanted to move some of the channels on 65535 to a different channel number. The same situation happens in the Freesat ARC, where there are multiple BBC1 channels on channel 101 for example. Pre-edits provide the means therefore to change the channel number on a specific ARC entry, before it is presented to the block list.

The logical sequence for Process_ARC is therefore:
  • Read the full ARC file
  • Extract the Bouquets
  • Pre-Edit any channel numbers in the selected Bouquets
  • Remove any channels which still match the block list
  • Update the LCF based on the resulting ARC Extract

You now have an LCF which is aligned to the final ARC Extract.

You can now make some changes to customise the LCF:
  • Exclude channels that you do not want to Sync into TVSource
  • Assign a different channel number to display in the guide(note the use of Assigned number as we still need to retain the original number to match future runs of Process_ARC)
  • Assign a different channel name to display on the guide
  • These changes are attributes of the channel in the LCF and will be maintained as the channel 'moves around' based on updates in the ARC

Finally you run SyncToTVSource which will align TVSource with the LCF for the tuners that you specify as 'in scope'.

I have taken time to answer your questions at length as I am sure this will help many other people understand what is going on, if they have not yet downloaded the package.

The package contains an extensive user manual and I hope these questions can also be answered from that!

LineUp has grown over many months into a fairly complex solution as I identified new scenarios to address. If you want to do something, it may be covered by one of the many parameters/options. Hopefully the parameter/option names will help you find the option you need!
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby MCE_DVB » Fri Oct 18, 2013 5:53 pm

Hi PRBUK,

You have indeed taken time to reply in full and the explanations help greatly in understanding the logic involved. The manual too is an excellent source of information and extremely well documented; I appreciate you exampling the use of parameters available and processes involved in greater depth here.

I hope to have some further time this weekend to finalise my set up and once in place I have no doubt will be a highly valued addition.

I will post back once I am set.

Many thanks again.

-LightsOn
MCE_DVB
 
Posts: 164
Joined: Tue Nov 01, 2011 2:29 pm

Re: LineUp - A Channel Line-Up Utility

Postby MCE_DVB » Sun Oct 20, 2013 12:31 pm

Hi PRBUK,

I am making good progress (one would hope so after nearly 30+ hours of playing and testing! all worth it to get everything just right though) goodness knows how many hours you put into development.

I have my sat. LCF and ARC and EPG (XMLTV) all working great – channel numbers etc all sorted. Very pleased thank you.

I then went to merge with the current FTA tuners – road block. Grrr. Your extra work once again comes into excellent play and I have followed the instructions. I manage to get channels into the LCF, make the changes so in essence I have an LCF I can sync like I do with the sat. set up. Not so for some reason. I have tried reset_LCF but get the below (test output on a random channel to test)

***** WARNING ***** Duplicate Source entries in Tuner #1 (299e Tuner 1) for nid(9018) tid(24640) sid(27712) name(Al Jazeera
Eng)

FAILED to add new channel to the lineup : ( ) 82 Al Jazeera Eng
***************************** Required Tuner Source Not Found ******************************
Unable to add this channel to the Lineup. This channel is not present on any In Scope Tuners.
Please perform a transponder scan for the In Scope Sources and Try Again!
82 Al Jazeera Eng was NOT added
**********************************************************************************************

I see two issues here I think:

1) It seems to think it cant tune to the channel?
2) It see’s multipal channels on the same tuner – true, but all three have the same SID,NID,Name etc so it can happily pick any, but won’t.

Hope you can help in my final push to get over this last hurdle 
MCE_DVB
 
Posts: 164
Joined: Tue Nov 01, 2011 2:29 pm

Re: LineUp - A Channel Line-Up Utility

Postby PRBUK » Sun Oct 20, 2013 5:51 pm

There should only be one channel with the same physicals, regardless of frequency. If there are duplicates within a tuner you get the 'duplicates' error message and the channel is not added by LineUp. You need to remove the duplicates either manually or via a rescan (easiest if you are using LineUp)..

If this is a DVBS tuner, remove the headend(where you select the satellite to scan), add it back and rescan. If DVBT, the fastest option is to delete the tuner and add it back. You will probably be left with only one channel with those physicals.

Astra recently shuffled their transponders and the result was a few transponders which duplicated each other for a transitional period. I posted a new 028.2 INI file which works with the full Sky ARC from which I had deleted those transponders I believed to be 'old'.

It looks like you are working with DVBT here. Try deleting and adding back the source with the same source name.
PRBUK
 
Posts: 152
Joined: Thu Dec 23, 2010 8:30 pm

Re: LineUp - A Channel Line-Up Utility

Postby MCE_DVB » Sun Oct 20, 2013 6:32 pm

HI PRBUK,

I'm still hard at it, thank you for the help.

So I too have today found that if I manually remove the duplicates it is then added. However I have to do this DVBLink side and this is lengthy and troublesome as one wrong click on delete and you have to start over. The only difference between the channels is the frequencies, all other details are the same, however I can't block by frequency as I understand it.

I updated the sat .ini you produced yesterday thanks to a pointer from oberon yesterday - thank you for this.

This particular issue is with DVBT tuners as you state and I have removed and added and scanned many many times today and I always get multiples that are identical. I am scanning full UK however but have to as using just a local signal misses nearly all channels.

The sat side is working a treat - the DVBT side is proving - to be honest - a total nightmare, its just not adding up which ever way I go about things. this is mainly a result of not being able to easily merge channels like in previous versions of dvblink. I wont be beaten here but I must finish this tonight or else I may be sleeping on the couch lol!
MCE_DVB
 
Posts: 164
Joined: Tue Nov 01, 2011 2:29 pm

PreviousNext

Return to Development corner

Who is online

Users browsing this forum: No registered users and 1 guest