How do I make my async coroutine function not block?

I’m trying to write some concurrent Python.

In my app below, I’m trying to subscribe to a websocket feed, but then continue through the app, so this code should just print hello. Instead, it blocks on feed.__init__.

I’m hoping I can build a websocket client to some update some application state.

How do I do feed.__init__ and continue running my program, while the websocket runs in the background?

def main():
  myFeed = feed('XBTUSD')
  while True:
    "hello"


import websockets
import asyncio
import json

class feed:
    def __init__(self, symbol):
        self.symbol = symbol
        self.uri = "wss://www.bitmex.com/realtime?subscribe=instrument,quote:{}".format(symbol)
        asyncio.get_event_loop().run_until_complete(self.socket())
        # asyncio.get_event_loop().run_forever()
   

    async def socket(self):
      async with websockets.connect(self.uri) as websocket:
        while True: 
          msg = await websocket.recv()
          self.process_msg(json.loads(msg))

    def process_msg(self, msg):
      try:
        data = msg['data'][0]
        if data['symbol'] == self.symbol:
            self.last_message = msg 

      except:
        print("Unhandled") 

Go to Source
Author: cjm2671

REMEMBER_COOKIE_DURATION in flask is not working

I am trying to configure my application to expire cookies after some amount of time however they are not working as expected, my cookies don’t get expired at all.

I have the following configuration in my app.py file:

from flask_login import LoginManager
from datetime import timedelta

app.config['REMEMBER_COOKIE_DURATION'] = timedelta(seconds=30)

login = LoginManager(app)
login.init_app(app)
login.login_view = 'login'

The following is my remember_me input:

<input type="checkbox" class="form-check-input" id="remember_me" name="remember_me">
<label class="form-check-label" for="remember_me">Remember me for 1 week</label>

Finally the following is my view function:

@app.route('/login', methods=['GET', 'POST'])
def login():
    if current_user.is_authenticated:
        return flask.redirect(flask.url_for('home'))
    if flask.request.method == 'POST':
        member_username = flask.request.form['member_username']
        member_password = flask.request.form['member_password']
        remember_me = flask.request.form.get('remember_me')

        existing_user = Users.query.filter_by(username=member_username).first()

        if existing_user is None or not existing_user.check_password(member_password):
            flask.flash('Invalid username or password')
            return flask.redirect(flask.url_for('login'))

        if existing_user:
            login_user(existing_user, remember=remember_me)

        next_page = flask.request.args.get('next')
        if not next_page or url_parse(next_page).netloc != '':
            next_page = flask.url_for('home')
        return flask.redirect(next_page)
    return flask.render_template('login.html')

The resources I have already followed:
https://code.luasoftware.com/tutorials/flask/how-to-configure-flask-login/
https://www.youtube.com/watch?v=CRvV9nFKoPI
flask-login: Chrome ignoring cookie expiration?

What am I missing?

Go to Source
Author: sherlock

How to deploy python microservices app updates?

I’m working on python microservices based app and I’m wondering how to ship updates.

Let’s start by a summary of what I have:

  • Code base on git (several repos)
  • The app is running on several remote raspberry pis
  • I access these via vpn network
  • Raspberry pis have a limited access to internet via GSM MtoM network
  • The main custom lib is installed with pip
  • Dependencies are installed at the first launch in house with good internet connection
  • The code of each service is ofuscated in place
    At the moment when I update my services I wrote a shell script which goes and download latest version for each service.
    It feels like it’s such a repetitive task for each device.

I only see 2 simple options:

  • connect to each device via vpn + ssh and run the update script (even an enhanced version)
  • At the end of the CI run a script that send latest.zip and create a service (script + cron ? python + systemd ) on each device that check for new .zip to update the services.
    Each of those options doesn’t look pretty clean.

What is usualy done in similar cases ?

Do you have any idea to share with me I don’t find anything that suits my needs.

Thanks for your inputs !

Go to Source
Author: AFZ84

redis connection error within docker container

I’m trying to setup a light-weight RQ task queue app running in Docker. I keep on getting this error:

redis.exceptions.ConnectionError: Error -2 connecting to redis:6379. Name or service not known.

or

ValueError: Redis URL must specify one of the following schemes (redis://, rediss://, unix://)
I get the first error if I hardcode redis://redis:6379 and the second when grabbing from environmental variable (os.getenv()). I looked at: link1 and link2, but there wasn’t really a solution, the thread went dead.

My redis connection code is like so:

import os
import redis
from rq import Worker, Queue, Connection

listen = ['default']
redis_url = os.getenv('REDIS_URL', 'redis://localhost:6379')

if __name__ == '__main__':
    print(f"redis url: {redis_url}")
    with Connection(redis.from_url(redis_url)):
        worker = Worker(list(map(Queue, listen)))
        worker.work()

When it prints the redis_url it is correct; I set the environmental var, REDIS_URL to redis://redis:6379 where redis is the hostname in docker-compose.yml.

  redis:
    image: redis:alpine
    expose:
      - '6379'

Appreciate any help. Thanks.

Go to Source
Author: Spencer Trinh

how to run the command pip3 install tensorflow on linux mint

i run the command above on linux mint and python3.
i get the error:

Failed building wheel for grpcio

and also

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=’/tmp/pip-build-lm1pg4la/grpcio/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘rn’, ‘n’);f.close();exec(compile(code, file, ‘exec’))” install –record /tmp/pip-r4j8ens0-record/install-record.txt –single-version-externally-managed –compile –user –prefix=” failed with error code 1 in /tmp/pip-build-lm1pg4la/grpcio/

please help me

Go to Source
Author: refaeldimri

Optimization for data storage

I’d like your advice on the design of my application.

I use websockets to receive new data and the request module to retrieve older data.
Then I use pyqtgraph to display data and tables etc with pyqt5.

There are some data that I don’t keep in memory, I just display them on screen without the possibility to interact with them, and I have other data that I keep in memory, with which I do some processing.

I would like to know if I should use dictionaries to store and process data or create a database with SQL or use pandas.
There will be a lot of inserting, extracting, deleting and a lot of calculations.

Potentially, when there are big moves, I could have thousands of messages per second to process, which I would have to add to my database, process and then display them on screen or do whatever I wanted with them.

If you have any questions, don’t hesitate.

Example of connection:

import websockets
import asyncio
import json

async def capture_data():
    subscriptions = [sub for sub in ["quote", "trade", "instrument"]]
    uri = "wss://www.bitmex.com/realtime?subscribe=" + ",".join(subscriptions)

    async with websockets.connect(uri) as websocket:
        while True:
            data = await websocket.recv()
            print(json.loads(data))


asyncio.get_event_loop().run_until_complete(capture_data())

Go to Source
Author: antho

i am getting an error of ‘turtle’ has no attribute ‘Turtle’

The mistakes done in the turtle library

1 : saving the file as “turtle.py”

2 : using the lower case for library as turtle.turtle()….the correct form is turtle.Turtle()

3 : if you have done the first mistake of saving the file as “turtle.py” don’t rename the file in the same folder …try a new place to save the file.

Go to Source
Author: Suchit Borge

Class Design question concerning adding a new “DRAFT” state to an object

Class Design question concerning adding a new “DRAFT” state to an object

We have some entities in our code:

-“View” and View has some attributes and also contains one or many “SubViews”

-And each “SubView” has some attributes and contains one or more “Tweet” entities.

So far so good. User edits Views/SubViews/Tweet (adding/removing tweets from subviews, adding/removing subviews fro views etc) in our UI and as everything is linked it updates fine. At the moment the user after editing, can decide to make her changes “Live” Or just discard them, there is no option to save and come later.

So now we want to introduce a “Draft” system. So user can edit a View/SubView/Tweet in a draft state (This is to be persisted) and eventually decide to make it “Live”. On making it “Live” the draft doesn’t die. User can continue to work on the draft and make it live again. So they both live side-by-side and have to be linked.

I’m not sure how to go about introducing this is code as the “Live” is just implicit in code/db atm. Some options I could think:

  1. Add a flag to each class with Live/Draft state indication. This will mean that I need to keep them in synch between the entities that are linked together
  2. Add another “Manager” like class that holds the objects related to Draft/Live versions of the views and manages the relationship between them. Using only the current types. The Manager is the only one that knows what is draft and what is live object
  3. Add a completely new type “DraftView”->”DraftSubView”->”DraftTweet” and manage the relationship between Draft and Live entities

At the DB level I’m thinking of just adding another column to the table to say Live/Draft.

More likely, there is a better way to do this and there are patterns to help with this, so i’m hoping I can get some opinions on my suggestions or better ways to do this. Hopefully I could make the question clear.

Thanks,

Go to Source
Author: user642770