TOOLSP
"
WELCOME !

Please ! Use [CODE] tags for your LINKS and CODE.
Favor usar balisas [CODE] para sus vínculos y código.
Merci d'utiliser les balises [CODE] pour vos liens et code.

[code]http://Thank.you[/code]
"
*** GITHUB ***
.
.
URL Resolver [GIT]
.
.
.
.
Jx Update [REDIRECT]
.
.
*** Associated ***
.
Pastebin Your list online
.
mediafire Upload Files
.
imgur Upload Pics
lyngsat TV logos collection
transparent .png
.
http://hola.org free? VPN
Hola.apk free? VPN App.
.
hidester- free Proxy
.
webgrabplus EPG - Eng
xmltv EPG - Fr
kazer EPG - Fr
.
.wordreference Traduction
.
mail.com (fast sign-in)
.
.
Latest topics
» SportsDevil back on Mega
Yesterday at 14:38 by skyhigh

» SportsDevil back on Mega
Fri 17 Nov - 5:30 by jairoxyz

» youtube playlists
Tue 14 Nov - 2:43 by jujuuj

» afromotion
Thu 9 Nov - 13:59 by doctor4979

» ? works on VLC, but not on kodi ?
Mon 6 Nov - 13:06 by jujuuj

» how make a request.post to get a repond
Tue 31 Oct - 16:01 by jujuuj

» how to play this in kodi
Tue 31 Oct - 15:21 by jujuuj

FRIENDS / PARTNERS

forum

créer un forum


help with multipage

View previous topic View next topic Go down

help with multipage

Post by oxus on Mon 29 May - 13:23

Hi
I use this code to try to add multipage to ustreamix.
The purpose is to use multiple search items.
Code is working but is getting links only from first page:
Code:
http://v2.ustreamix.com/cr.php?q=dolce
Can you help me with this?
Thanks!
Code:

<item>
<title>Ustreamix.com RO</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR yellow][makelist.param2][/COLOR] [COLOR dodgerblue]([makelist.param3])[/COLOR]</title>
<link>$doregex[getUrl]|User-Agent=iPad</link>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re, requests
def GetLSProData(page_data,Cookie_Jar,m):
 headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux i686; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0'}
 data = []
 sites = ['http://v2.ustreamix.com/cr.php?q=dolce', 'http://v2.ustreamix.com/cr.php?q=digi', 'http://v2.ustreamix.com/cr.php?q=romania']

 for site in sites:
    source = requests.get(site, headers=headers).text
    data += re.findall('(?s)<p><a href="(.*?)".*?blank">(.*?)\s*<.*?live">(.*?)<', source)
 return data
]]></expres>
<page></page>
</regex>


<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
import re, requests
def GetLSProData(page_data,Cookie_Jar,m,):
  first_ip = re.findall("x_first_ip.+?'(.*?)'", page_data)[0]
  first_c = re.findall("x_first_c.+?'(.*?)'", page_data)[0]
  ket2 = re.findall("ket2.+?'(.*?)'", page_data)[0]
  first_ua = re.findall("x_first_ua.+?'(.*?)'", page_data)[0]
  preurl = 'http://tmg.ustreamix.com/'  # here is the server url which changed most of time
  url = preurl + 'stats.php?p=' + first_ip + '&C=' + first_c + '&Ket=' + ket2
  source = requests.get(url, headers = {'Referer': 'http://v2.ustreamix.com/[makelist.param1]', 'User-Agent': first_ua, 'Accept': '*/*'}).text
  tok = re.findall('var jdtk="(.*?)"', source)[0]
  return preurl + 'tmg.m3u8?' + '[makelist.param1]' + '&token=' + tok
]]></expres>
<page>http://v2.ustreamix.com/[makelist.param1]</page>
<agent>Mozilla/5.0 (X11; Linux i686; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0</agent>
<cookieJar></cookieJar>
</regex>
</item>

oxus

Messages : 8
Date d'inscription : 2017-04-17

View user profile

Back to top Go down

Re: help with multipage

Post by JonnyB on Mon 29 May - 15:27

Hi,

I think the site has a timeout between searches...

But maybe it's possible to find an expres that matches the results, this one matches digi and dolce but I don't know how to add the other, someone with more skills should know:

Code:
<item>
<title>Ustreamix.com RO</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR yellow][makelist.param2][/COLOR] [COLOR dodgerblue]([makelist.param3])[/COLOR]</title>
<link>$doregex[getUrl]|User-Agent=iPad</link>
]]></listrepeat>
<expres><![CDATA[<a href="([\s\S]{0,50})"\s+target[\s\S]{0,20}>((?:(?i)digi|dolce).*?)<[\s\S]{0,20}live">(.*?)<]]></expres>
<page>http://v2.ustreamix.com/</page>
</regex>


<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
import re, requests
def GetLSProData(page_data,Cookie_Jar,m,):
  first_ip = re.findall("x_first_ip.+?'(.*?)'", page_data)[0]
  first_c = re.findall("x_first_c.+?'(.*?)'", page_data)[0]
  ket2 = re.findall("ket2.+?'(.*?)'", page_data)[0]
  first_ua = re.findall("x_first_ua.+?'(.*?)'", page_data)[0]
  preurl = 'http://tmg.ustreamix.com/'  # here is the server url which changed most of time
  url = preurl + 'stats.php?p=' + first_ip + '&C=' + first_c + '&Ket=' + ket2
  source = requests.get(url, headers = {'Referer': 'http://v2.ustreamix.com/[makelist.param1]', 'User-Agent': first_ua, 'Accept': '*/*'}).text
  tok = re.findall('var jdtk="(.*?)"', source)[0]
  return preurl + 'tmg.m3u8?' + '[makelist.param1]' + '&token=' + tok
]]></expres>
<page>http://v2.ustreamix.com/[makelist.param1]</page>
<agent>Mozilla/5.0 (X11; Linux i686; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0</agent>
<cookieJar></cookieJar>
</regex>
</item>

JonnyB

Messages : 9
Date d'inscription : 2017-04-18

View user profile

Back to top Go down

Re: help with multipage

Post by twogun on Mon 29 May - 20:09

@oxus
@JonnyB is correct.  you need to delay the loop before it continues.  here is your code with a 5 second delay added.  This seems to work.  You can try reducing the wait time to speed it up but be careful not to set it to aggressively or it will not return data.

Code:


    time.sleep(5) # delays for 5 seconds

This will be slow to load. it takes up 15 seconds just waiting.


Last edited by twogun on Thu 22 Jun - 18:50; edited 1 time in total
avatar
twogun

Messages : 33
Date d'inscription : 2017-04-03

View user profile

Back to top Go down

Re: help with multipage

Post by geetee on Mon 29 May - 22:20

Works really well for me, thanks

geetee

Messages : 7
Date d'inscription : 2017-05-28

View user profile

Back to top Go down

Re: help with multipage

Post by solved on Tue 27 Jun - 0:42

.

solved

Messages : 35
Date d'inscription : 2017-06-27

View user profile

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum