Extract data from web with google script

I’m trying to use the function below (source) to avoid using importxml in Google Sheets:

function importRegex(url, regex_string) {
  var html, content = '';
  var response = UrlFetchApp.fetch(url);
  if (response) {
    html = response.getContentText();
    if (html.length && regex_string.length) {
      var regex = new RegExp( regex_string, "i" );
      content = html.match(regex)[1];
    }
  }
  Utilities.sleep(1000);
  return content;  
}

I want to extract stock prices and day change from Yahoo Finance, for example from: https://finance.yahoo.com/quote/TSLA/, but I can’t get the regex_string right. I have tried several strings but none seens to work. Some one can help me please?

=importRegex("https://finance.yahoo.com/quote/TSLA/", "<span class=""Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)"" data-reactid=""32"">(.*)</span>")

Go to Source
Author: Caseyblack999

Best archtitecture and methods for high performance computing that needs to scale

I have to make a decision regarding architecture and methods for the rewrite of a proof of concept application I wrote 10 years ago in c++….

It’s about high performance position calculation based on multi-trilateration.
Hunderts, thousands of IoT Sensors are sending it’s JSON based distance information to a host by using MQTT. From there the information needs to be processed.

My goal is to rewrite it, so it will get more real-time, scalable and run the position-solver-application in the cloud or on-premises with utilizing the cpu as efficient as possible by using all of the cores / threads.

If you start from scratch which architecture, language and methods would you use?
E.g.

GoLang ? C++ with threads? Rust? Python?
Architecture ?
Docker?
GPU support?

some metrics:
up to 10.000 sensors are sending distance 200 JSON messages per second to the MQTT Broker

(In my proof of concept there were just 20 sensors and 5 messages per second)

Any recommendation?

Will be a open-source project by the way.

Best regards,
//E

Go to Source
Author: Ersan

How to test C++ code when using C functions

I wrote a c++ class, this class uses the c function library written by a third party.

extern "C" {
#include "nats/nats.h"
}

class NatsConnection {
  public:
  void Connect() {
    natsConnection_Connect(&natsConnection_, natsOptions_);
    natsConnection_SubscribeSync(&natsSubscription_, natsConnection_,
                                 configuration_.subject.c_str());
    // some other c++ code.
  }
}

The above class uses functions imported from c: natsConnection_Connect, natsConnection_SubscribeSync.

Now I need to write a unit test to cover some other c++ code, I am using gtest, I know how to mock a C++ class, but once I use the C code I don’t know how to start.

how do i write the test? Is there has a best practice?

Go to Source
Author: MengHan Yu

How to handle different types of errors in Clean Architecture?

So, in the process of creating a user there are 4 possible outcomes:

  1. Username is already taken

  2. Email is already taken

  3. Username is invalid
  4. Email is invalid

Here is what I have in the controller for now:

  const user = await this.addUserService.add(username, email, password)

  if (user) {
    return {
      statusCode: 201,
      body: user
    }
  } else {
    return {
      statusCode: 400,
      body: // ???
    }
  }

The thing is, what should the addUserService return if the username/email is invalid or in use?

I am thinking about creating different error types, such as InvalidParamsError, ParamInUseError and share them between the UserController and UserService.

So I would have something like this:

  const result = await this.addUserService.add(username, email, password)

  if (typeof result === InvalidParamsError || typeof result === ParamInUseError) {
    return {
      statusCode: 401,
      body: result.message
    }
  } else { // then we assume user has been created
    return {
      statusCode: 201,
      body: 'User created!'
    }
  }

Is it okay to share these Error classes between the controller and the service? If not, or even if it’s okay, is there a better way to handle errors when the outcome is not a simple true or false?

Thanks.

Go to Source
Author: kibe

File which contains table_names and column_names in it. Now i want to pull this two values into two variables and loop them

I have file called objects.txt contains tables and columns as below.

Part:1
Table_name: customer1
Column_name: account_no

Part:2
Table_name: customer2
Column_name: purchase_id
.
.
Part:n

Now, I want to pick this table_name and column_name from the file as two variables in a loop and send to the sqlplus to run the below query.

For tab in grep of table_name
For col in grep of column_name

Set table=$tab
Set column=$col
Set Null_check='sqlplus -s/@$oracle' ‹‹ENDOFSQL
set serveroutput on;
Select count(*) from (select $column from $table where $column is null);
Exit
ENDOFSQL

Please help me how to loop the this two variables in single for loop iteration and send to this SQL query

Go to Source
Author: dinesh valavala

How to load new text widgets and polls into the sidebar without reloading the page?

We are creating a WordPress website page for the online broadcast of the conference. The page should work so that when a site visitor opens this page, the content on this page is automatically loaded without reloading the page itself.

In the right column there will be this chat “Wise Chat Pro”.
In the central column there will be a video player to watch the broadcast from YouTube, and under it will be a block where you can ask the speaker of the conference a question and the questions themselves will automatically appear without reloading the page, as in this example: https: //themes.webberwebber. com / live-neutral / live-relation-directly-from-an-imporant-event /. How to implement this is understandable, using the comments on the page and automatically load them by ajax

The problem is that in the left column of the site, as well as at the bottom of the central column, when a new quote or survey is published, they are automatically loaded on the page, without reloading the page itself.

Is it possible to load new quotes and polls into the sidebar of the WordPress site without reloading the page? I would be very grateful for a hint in which direction to move and how this can be implemented.

enter image description here

Go to Source
Author: Danil Pridvorov

How to keep secrets out of version control with kustomize?

I’ve started using kustomize. It lets you generate secrets with something like:

secretGenerator:
  - name: mariadb-env
    envs:
      - mariadb.env

This is great because kustomize appends a hash so that every time I edit my secret, kubernetes will see it as being new and restart the server.

However, if I put kustomization.yaml under version control, then it kind of entails that I put mariadb.env under version control too. If I don’t, then kubernetes build x will fail because of the missing file [for anyone that tries to clone the repo]. Even if I don’t put it under VCS, it still means I have these secret files on my dev workstation.

Prior to adopting kustomize, I’d just create the secret once, send it to the kubernetes cluster, and let it live there. I could still reference in my configs by name, but with the hash, I can’t really do that anymore. But the hash is also incredibly useful for forcing the restart.

How are people dealing with this?

Go to Source
Author: mpen

How to install and configure pNFS Servers, MetaData Servers, and clients on Ubuntu 20.04?

Does Ubuntu20.04 support pNFS Server(Data and MetaData) and Client side of pNFS?

If YES, Then How to install and configure it? Is there any comprehensive guide?

How to install and configure pNFS Servers, MetaData Servers, and clients on Ubuntu 20.04?

Go to Source
Author: Milad

MySQL InnoDB Weird Query Performance

I designed two tables, both using InnoDB.

The first one has columns “Offset, Size, and ColumnToBeIndexed” with BIGINT “Offset” being the primary key, and the second one has columns “OffsetAndSize, and ColumnToBeIndexed” with BIGINT “OffsetAndSize” being the primary key. And there are both 15 millions rows in each table.

I added indexes to both tables on “ColumnToBeIndexed.

My two queries for them are

SELECT Offset, Size 
FROM someTable 
WHERE ColumnToBeIndex BETWEEN 20 AND 10000 
ORDER BY Offset ASC

and

SELECT OffsetAndSize 
FROM someTable 
WHERE ColumnToBeIndex BETWEEN 20 AND 10000 
ORDER BY OffsetAndSize ASC

Because the second query could use the secondary index and does not need to look up the clustered index to find the “size” information, I expected that the second query on the second table performs better than the first query on the first table. However, as the test came out, it turned out that the first query performs better every single time.

Does anybody out there know what seems to be the problem?

Go to Source
Author: Bruce

Preserve leading zeros in a Google Sheets CONCATENATE function

I am trying to create about 50 cells that have number combinations which look like variations of this:

01:05:39

I am using a RANDBETWEEN function to generate three pairs of numbers (e.g. =RANDBETWEEN(0,59) in cells A1, B1, and C1, and then using CONCATENATE (e.g. =CONCATENATE(A1, ":", B1, ":", C1) ) to put them together in another cell. I’ve tried a number of different methods (including Plain Text formatting on the RANDBETWEEN cells, pasting values only into a different column and using those values, wrapping parts of the formula with a TEXT function, custom formatting with 00:00:00 on the CONCATENATE cells, etc.), but my resulting cells are not retaining the leading zeros. They end up looking like this:

1:5:39

If you’re wondering why I’m trying to do this, I’m a product designer and I need to make some realistic looking timestamps for a mockup. Anyone know how to solve this? Maybe I’m overcomplicating things somehow and there’s a much easier way?

Go to Source
Author: Peter Bentley

How to manage user java deployments in a small team, across different java applications (which require different java versions)?

I provide a java application (desktop client) which requires jdk11 (it really does).
We installed that SDK on all users and were happy for a while.

Then, someone shared with us his java application (also a desktop client, but unrelated to mine), which only works with jre8 (he is not willing to recompile to 11 and his status in the company is such that we must conform to him at all costs)…

I’ve instructed the team members to change their JAVA_HOME between an installed jre8 and jdk11, as they need, but this doesn’t seem like more than an annoying manual stop gap.

I am wondering what is a good, modern approach to allow, for example: a system-wide jre8 for the applications I don’t control but also code a little something into my project that can switch to jdk11 for the duration of their use of my application?

Since I am delivering a project with source code (which compiles every time they run, because they sometimes need to modify some of the java code), I am thinking along the lines of including a portable jdk11 and have the project refer to that as the SDK, relatively (so that they don’t have to install anything to get my application going).

I actually tried this a little bit but couldn’t get it done and I am wondering if it’s a good approach that is worth fixing or there is a better idea?

Can you share your experience about this relatively common problem?

Edit: + I am not sure a containerized solution is viable. It sounds like I’ll have to help them set up their system to support that and then wrap my project in a container – which sounds like a can of worms for someone with my skills (middleweight java dev, at best)

Go to Source
Author: DraxDomax

Ubuntu 20.04 takes too long to boot

I have recently installed Ubuntu 20.04 on my HP-notebook(intel core i-5, 8GB RAM, AMD Radeon 2GB). It takes more than 2 min to reach the login screen.

I used the command dmesg to analyze which process it taking too much time only too see the below output:

If you go to timestamp [ 67.555730] rfkill: input handler disabled] , the process just after that by the name of audit has been called so many times and took nearly 50 secs of the boot process.

[   67.460851] [drm] ib test on ring 1 succeeded in 0 usecs
[   67.460874] [drm] ib test on ring 2 succeeded in 0 usecs
[   67.460899] [drm] ib test on ring 3 succeeded in 0 usecs
[   67.460924] [drm] ib test on ring 4 succeeded in 0 usecs
**[   67.555730] rfkill: input handler disabled**
[  111.331719] audit: type=1400 audit(1591675362.633:47): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine" pid=1687 comm="apparmor_parser"
[  111.332012] audit: type=1400 audit(1591675362.633:48): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=1687 comm="apparmor_parser"
[  112.182135] audit: type=1400 audit(1591675363.485:49): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap-update-ns.snap-store" pid=1689 comm="apparmor_parser"
[  112.188690] audit: type=1400 audit(1591675363.493:50): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap-update-ns.canonical-livepatch" pid=1691 comm="apparmor_parser"
[  112.190389] audit: type=1400 audit(1591675363.493:51): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.canonical-livepatch.hook.configure" pid=1694 comm="apparmor_parser"
[  112.192167] audit: type=1400 audit(1591675363.497:52): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.canonical-livepatch.canonical-livepatch" pid=1692 comm="apparmor_parser"
[  112.192408] audit: type=1400 audit(1591675363.497:53): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.canonical-livepatch.canonical-livepatchd" pid=1693 comm="apparmor_parser"
[  112.195750] audit: type=1400 audit(1591675363.497:54): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap-update-ns.vlc" pid=1698 comm="apparmor_parser"
[  112.196569] audit: type=1400 audit(1591675363.501:55): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.snap-store.snap-store" pid=1695 comm="apparmor_parser"
[  112.197056] audit: type=1400 audit(1591675363.501:56): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap.snap-store.ubuntu-software" pid=1696 comm="apparmor_parser"
[  118.265070] kauditd_printk_skb: 2 callbacks suppressed
[  118.265076] audit: type=1400 audit(1591675369.569:59): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/snap/core/9289/usr/lib/snapd/snap-confine" pid=1796 comm="apparmor_parser"
[  118.265095] audit: type=1400 audit(1591675369.569:60): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/snap/core/9289/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=1796 comm="apparmor_parser"
[  118.797549] audit: type=1400 audit(1591675370.101:61): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine" pid=1798 comm="apparmor_parser"
[  118.836034] audit: type=1400 audit(1591675370.141:62): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=1798 comm="apparmor_parser"
[  119.433628] audit: type=1400 audit(1591675370.737:63): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap-update-ns.core" pid=1800 comm="apparmor_parser"
[  119.529999] audit: type=1400 audit(1591675370.833:64): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.core.hook.configure" pid=1801 comm="apparmor_parser"
[  120.144506] audit: type=1400 audit(1591675371.449:65): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.snap-store" pid=1803 comm="apparmor_parser"
[  120.175883] audit: type=1400 audit(1591675371.481:66): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap-update-ns.vlc" pid=1806 comm="apparmor_parser"
[  120.190680] audit: type=1400 audit(1591675371.493:67): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.ubuntu-software" pid=1804 comm="apparmor_parser"
[  120.211819] audit: type=1400 audit(1591675371.517:68): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap-update-ns.canonical-livepatch" pid=1808 comm="apparmor_parser"
[  147.119565] kauditd_printk_skb: 6 callbacks suppressed
[  147.119569] audit: type=1400 audit(1591675398.426:75): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine" pid=1934 comm="apparmor_parser"
[  147.119577] audit: type=1400 audit(1591675398.426:76): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/snap/snapd/7777/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=1934 comm="apparmor_parser"
[  147.918540] audit: type=1400 audit(1591675399.222:77): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.ubuntu-software" pid=1938 comm="apparmor_parser"
[  147.948852] audit: type=1400 audit(1591675399.254:78): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.ubuntu-software-local-file" pid=1939 comm="apparmor_parser"
[  148.020594] audit: type=1400 audit(1591675399.326:79): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.snap-store" pid=1937 comm="apparmor_parser"
[  148.216393] audit: type=1400 audit(1591675399.522:80): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap-update-ns.snap-store" pid=1936 comm="apparmor_parser"
[  210.116607] [drm] PCIE gen 3 link speeds already enabled
[  210.133629] [drm] PCIE GART of 2048M enabled (table at 0x0000000000040000).
[  210.133736] radeon 0000:01:00.0: WB enabled
[  210.133739] radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000080000c00 and cpu addr 0x0000000002c29bac
[  210.133740] radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000080000c04 and cpu addr 0x00000000f44304ae
[  210.133742] radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000080000c08 and cpu addr 0x0000000031bde5cd
[  210.133743] radeon 0000:01:00.0: fence driver on ring 3 use gpu addr 0x0000000080000c0c and cpu addr 0x00000000d7e92f5c
[  210.133744] radeon 0000:01:00.0: fence driver on ring 4 use gpu addr 0x0000000080000c10 and cpu addr 0x000000001f77e5bc
[  210.326579] [drm] ring test on 0 succeeded in 1 usecs
[  210.326584] [drm] ring test on 1 succeeded in 1 usecs
[  210.326588] [drm] ring test on 2 succeeded in 1 usecs
[  210.326596] [drm] ring test on 3 succeeded in 4 usecs
[  210.326603] [drm] ring test on 4 succeeded in 4 usecs
[  210.326671] [drm] ib test on ring 0 succeeded in 0 usecs
[  210.326732] [drm] ib test on ring 1 succeeded in 0 usecs
[  210.326820] [drm] ib test on ring 2 succeeded in 0 usecs
[  210.326833] [drm] ib test on ring 3 succeeded in 0 usecs
[  210.326844] [drm] ib test on ring 4 succeeded in 0 usecs
[  210.334790] rfkill: input handler enabled
[  210.715243] Bluetooth: RFCOMM TTY layer initialized
[  210.715258] Bluetooth: RFCOMM socket layer initialized
[  210.715273] Bluetooth: RFCOMM ver 1.11
[  224.498251] audit: type=1400 audit(1591675475.797:81): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.ubuntu-software-local-file" pid=2256 comm="apparmor_parser"
[  224.529105] audit: type=1400 audit(1591675475.829:82): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.snap-store" pid=2254 comm="apparmor_parser"
[  224.546528] audit: type=1400 audit(1591675475.845:83): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="snap.snap-store.ubuntu-software" pid=2255 comm="apparmor_parser"
[  224.820589] audit: type=1400 audit(1591675476.121:84): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="snap-update-ns.snap-store" pid=2253 comm="apparmor_parser"
[  233.396341] rfkill: input handler disabled
[  236.218843] [drm] PCIE gen 3 link speeds already enabled
[  236.230351] [drm] PCIE GART of 2048M enabled (table at 0x0000000000040000).
[  236.230455] radeon 0000:01:00.0: WB enabled
[  236.230458] radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000080000c00 and cpu addr 0x0000000002c29bac
[  236.230459] radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000080000c04 and cpu addr 0x00000000f44304ae
[  236.230460] radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000080000c08 and cpu addr 0x0000000031bde5cd
[  236.230461] radeon 0000:01:00.0: fence driver on ring 3 use gpu addr 0x0000000080000c0c and cpu addr 0x00000000d7e92f5c
[  236.230462] radeon 0000:01:00.0: fence driver on ring 4 use gpu addr 0x0000000080000c10 and cpu addr 0x000000001f77e5bc
[  236.422306] [drm] ring test on 0 succeeded in 1 usecs
[  236.422311] [drm] ring test on 1 succeeded in 1 usecs
[  236.422315] [drm] ring test on 2 succeeded in 1 usecs
[  236.422323] [drm] ring test on 3 succeeded in 4 usecs
[  236.422329] [drm] ring test on 4 succeeded in 4 usecs
[  236.422393] [drm] ib test on ring 0 succeeded in 0 usecs
[  236.422422] [drm] ib test on ring 1 succeeded in 0 usecs
[  236.422554] [drm] ib test on ring 2 succeeded in 0 usecs
[  236.422583] [drm] ib test on ring 3 succeeded in 0 usecs
[  236.422597] [drm] ib test on ring 4 succeeded in 0 usecs

Also, it takes about 10 secs to load the desktop and icons from login screen.

Below is the head of the output of $system-analyze blame.

41.797s plymouth-quit-wait.service                           
25.586s man-db.service                                       
21.755s NetworkManager-wait-online.service                   
19.547s networkd-dispatcher.service                          
12.294s apt-daily.service                                    
10.432s dev-sda2.device                                      
10.193s udisks2.service                                      
 9.942s accounts-daemon.service                              
 7.864s NetworkManager.service                               
 7.050s snapd.service                                        

Go to Source
Author: Hades

Where would the Jenkins job run if I don’t mention any Agent labels?

I my Jenkins setup I have a few set of jobs which uses agent to build and deploy. I achieved it by mentioning the agent label in the job configurations.

I have one job which has to execute on the master jenkins. If i don’t mention any label in job configuration and if no agents are online, the job gets executed on master jenkins. If any agent is online, instead of executing in master jenkins, it picks up some random agent and executes the job. How can i execute it on master even if agents are online?

Go to Source
Author: B. Akshay