x

Can breathing exercises really help protect you from covid-19?

Taking deep breaths and forcing a cough can help clear mucus, but these techniques are unlikely to prevent or treat coronavirus infections – here’s why




x

Ripples in Earth’s atmosphere make distant galaxies appear to flash

Faraway galaxies have been spotted unexpectedly flashing up to 100 times their usual brightness, and it seems to be caused by eddies in Earth’s atmosphere




x

Electrical devices implanted in the brain may help treat anorexia

In a small trial, implanting electrodes into the brain helped women with severe anorexia gain weight and feel less anxious and depressed




x

Weird radio signals spotted in our galaxy could solve a space mystery

Weird blasts of radio waves from space called fast radio bursts have been baffling astronomers since they were discovered, but after finding one in our galaxy we may finally know what creates them




x

SpaceX mission control to do social distancing for first crewed flight

SpaceX’s first crewed launch is planned for 27 May and will be run from a mission control with desks set six feet apart to comply with social distancing protocols




x

We really do relive experiences from waking life when we sleep

Brain implants have revealed that we replay conscious experiences while we sleep, with the same patterns of neurons firing during sleep as in waking life




x

Old Familiar Faces XVIII pt 1




x

Old Familiar Faces XVIII pt 2




x

Old Familiar Faces XVIII pt 3




x

Extended Stay




x

Yes, the dam is breaking. The exploding reaction to the polyam 'House Hunters' episode




x

Friday Polynews Roundup — Not all polyfamilies are FMF throuples, upcoming in TV and film, and a future of extended chosen family.




x

Friday Polynews Roundup — Safer sex in the pandemic. Move a metamour in for the duration? Skills for bottled-together partners, and more.




x

Friday Polynews Roundup — When this isolation ends, good long-distance sex, how to open a relationship, and more.




x

They're noticing us: "Multi-Partner Sexual-Rights Crusade on the Horizon"






x

How io_uring and eBPF Will Revolutionize Programming in Linux

#263 — April 22, 2020

Read on the Web

StatusCode Weekly
Covering the week's news in software development, ops, platforms, and tooling.

The Devastating Decline of a Brilliant Young Coder — This is not a technical article but is an important one nonetheless. Lee Holloway essentially programmed Cloudflare into being. But then he became distant and unpredictable, and what happened to him is something that could affect any of us ????

Sandra Upson (WIRED)

How io_uring and eBPF Will Revolutionize Programming in Linux — Even more exciting times are coming for development on Linux thanks to these technologies. A good overview.

Glauber Costa

Slow CI Build? Get a 41:1 ROI by Switching to Semaphore — For every $1 invested in Semaphore, engineers gain $41 in reclaimed productivity. Who said money can’t buy you time?

Semaphore 2.0 sponsor

▶  Mob Programming and the Power of Flow — I enjoyed this insightful walk through the idea of bringing people together and attempting to develop things in an efficient way with numerous people around the same machine (a.k.a. ‘mob’ programming). It’s not for everyone, but it’s neat to see how it can work.

Woody Zuill

Cloudflare Workers Now Supports.. COBOL — COBOL is one of the earliest things you could really call a programming language (it first appeared in 1959!) and is often a source of amusement because it’s seen as old, verbose, clunky, and difficult to maintain. Nonetheless, it’s still in use (particularly in legacy systems) and you can use with Cloudflare Workers too!

John Graham-Cumming

Quick bytes:

???? Jobs

DevOps Engineer at X-Team (Remote) — Join the most energizing community for developers. Work from anywhere with the world's leading brands.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Stories and Opinions

How 'Memories', a 256 Byte Demo, Was Coded — You can watch the demo here or enjoy learning just how these unusual developers cram so much into so little space.

HellMood/DESiRE

The Computer Scientist Who Can’t Stop Telling Stories — For pioneering computer scientist Donald Knuth, good coding is synonymous with beautiful expression.

Quanta Magazine

▶  Discussing NGINX and Service Meshes with Alan Murphy — I enjoyed this SE Daily episode last week and learnt a fair bit.

Software Engineering Daily podcast

End-to-End Observability for Microservice Environments — Optimize service costs and reduce MTTR with full data correlation, payload visibility and automated tracing. Try free.

Epsagon sponsor

▶  Performance Profiling for Web Applications — An overview of how to use Chrome DevTools to understand a Web application’s performance bottlenecks.

Sam Saccone

Are Object Stores Starting to Look Like Databases? — A bit, yes.

Alex Woodie (Datanami)

The Case Against CS Master’s Degrees

Oz Onay

Why I Stopped Using Microservices

Robin Wieruch

???? Tutorials

Ask HN: I'm A Software Engineer Going Blind, How Should I Prepare? — This is something I hope none of you have to go through, but we’ve linked to other stories about being a blind coder in the past, and some form of sight loss will affect many of us over the years.

Hacker News

Writing an 'Emulator' in JavaScript (and Interfacing with Multiple UIs) — Tania built a Chip-8 interpreter in JavaScript and has gone into quite a bit of detail about what was involved here. Lots of neat bits and pieces to pick up from this.

Tania Rascia

What It Took to Build a Serverless App That Texts Positive COVID-19 News — Code, a screencast tour, and an article looking at what it took to build a simple serverless app using C#, Azure Functions, and Twilio to text news alerts (but only ones with positive sentiments!)

Gwyneth Pena S.

If You Use grep On Text Files, Use the -a (--text) Option — I could explain why but then you wouldn’t need to read this. Makes a good point.

Chris Siebenmann

Event-Reduce: An Algorithm to Optimize Frequently Running Queries? — In brief, the idea is that rather than having to re-run queries when data changes on a table, you can basically merge in changes to previous query results. Be sure to check the FAQs.

Daniel Meyer

Embedding Binary Objects in C

Ted Unangst

???? Code and Tools

Desed: A Debugger for sed — Demystify and debug your sed (the text processor that comes with nearly every Unix) scripts, from the comfort of your terminal. Step through line by line, place breakpoints, etc.

SoptikHa2

Falcon: An Open-Source, Cross Platform SQL Client — Built around Electron and React, this basic client can quickly do chart visualizations of query results and can connect to RedShift, MySQL, PostgreSQL, IBM DB2, Impala, MS SQL, Oracle, SQLite and more.

Plotly

The SaaS CTO Security Checklist

Sqreen sponsor

Termible: Offer Terminal Apps in the Browser Without Installation — This is a commercial service but I find the idea intriguing. You provide a Dockerfile, embed some code on your site, and let people play with your product/service “live”. HTTPie seems to use it for its live examples.

Termible

X410: An X Server for Windows 10 — If you’re using WSL (Windows Subsystem for Linux) to run Linux behind the scenes of a Windows 10 install, X410 takes things to another graphical level.

Choung Networks

60 Linux Networking Commands and Scripts“I decided to create a network tools go-to-list for myself. Then, I thought, why not turn the list into a blog post?”

Hayden James

Brök: A Tool to Find Broken Links in Text Documents — Built in Haskell.

Mark Wales

xsv: A Fast CSV Command Line Toolkit Written in Rust. — Another ‘Swiss Army knife’ for your slightly structured data.

Andrew Gallant




x

Gavin Newsom Signs Executive Order to Mail Every Voter a Ballot for November Elections

California Governor Gavin Newsom announced Friday that he had signed an executive order to mail ballots to the state’s 20.6 million registered voters, citing potential health risks due to the ongoing coronavirus pandemic.“There’s a lot of excitement around this November’s election in terms of making sure that you can conduct yourself in a safe way, and make sure your health is protected,” Newsom said Friday. In March, the state allowed ballots to be mailed in for its primary, which saw a record-high of 72 percent of all ballots that were cast by mail.California Secretary of State Alex Padilla, who heads the state’s elections, commended the move “It’s great for public health, it’s great for voting rights, it’s going to be great for participation,” he said. California already allows for generous absentee voting, passing a 2002 decision which gives voters the option to request permanent voting by mail, regardless of the reason.While Newsom’s decision applies only to the November election, it could set a precedent for other Democratic states, with voting by mail quickly becoming a partisan issue. It comes after the state’s lawmakers and local officials requested the measure, saying coronavirus will severely hamper voting efforts, a complaint echoed by prominent Democrats.“Why should we be saying to people, ‘Stand in line for hours,’ when we don’t even want you leaving the house?” House Speaker Nancy Pelosi said in April. But President Trump has repeatedly slammed calls for mail-in voting, saying last month that it lets “people cheat” and involves “a lot of dishonesty.”Newsom said that his order would still allow an “appropriate number” of in-person voting sites, saying that some voters, including those that are disabled, require technological help to cast a ballot.





x

20,000 migrants have been expelled along border under coronavirus order

More than 90% of the families, children and single adults that Border Patrol encountered in April were swiftly expelled under a public health order.





x

The 'mind-blowing' story of the ex-Green Beret who tried to oust Venezuela's Maduro

Jordan Goudreau once pushed a plan to protect U.S. schools. Then he moved on to a more daring pursuit, which also didn't end well.





x

Texas governor amends lockdown and orders salon owner freed from jail

The governor's order names the Dallas hairdresser who was jailed on Tuesday for staying open.





x

A 1996 court declaration written by Tara Reade's ex-husband shows she spoke of harassment in Biden's Senate office

"It was obvious that this event had a very traumatic effect on (Reade), and that she is still sensitive and effected (sic) by it today," Dronen wrote.





x

3 nurses strangled in Mexico; border mayor gets coronavirus

Three sisters who worked in Mexico's government hospital system were found murdered by strangling, authorities in the northern border state of Coahuila announced Friday, stirring new alarm in a country where attacks on health care workers have occurred across the nation amid the coronavirus outbreak. Two of the sisters were nurses for the Mexican Social Security Institute and the third was a hospital administrator, but there was no immediate evidence the attack was related to their work. The National Union of Social Security Employees called the killings “outrageous and incomprehensible.”





x

Report says cellphone data suggests October shutdown at Wuhan lab, but experts are skeptical

U.S. and U.K. intel agencies are reviewing the private report, but intel analysts examined and couldn't confirm a similar theory previously.





x

'Never Seen Anything Like This': Experts Question Dropping of Flynn Prosecution

WASHINGTON -- The Justice Department's decision to drop the criminal case against Michael Flynn, President Donald Trump's former national security adviser, even though he had twice pleaded guilty to lying to investigators, was extraordinary and had no obvious precedent, a range of criminal law specialists said Thursday."I've been practicing for more time than I care to admit and I've never seen anything like this," said Julie O'Sullivan, a former federal prosecutor who now teaches criminal law at Georgetown University.The move is the latest in a series that the department, under Attorney General William Barr, has taken to undermine and dismantle the work of the investigators and prosecutors who scrutinized Russia's 2016 election interference operation and its links to people associated with the Trump campaign.The case against Flynn for lying to the FBI about his conversations with the Russian ambassador was brought by the office of the former special counsel, Robert Mueller. It had become a political cause for Trump and his supporters, and the president had signaled that he was considering a pardon once Flynn was sentenced. But Barr instead abruptly short-circuited the case.On Thursday, Timothy Shea, the interim U.S. attorney in the District of Columbia, told the judge overseeing the case, Emmet G. Sullivan, that prosecutors were withdrawing the case. They were doing so, he said, because the department could not prove to a jury that Flynn's admitted lies to the FBI about his conversations with the ambassador were "material" ones.The move essentially erases Flynn's guilty pleas. Because he was never sentenced and the government is unwilling to pursue the matter further, the prosecution is virtually certain to end, although the judge must still decide whether to grant the department's request to dismiss it "with prejudice," meaning it could not be refiled in the future.A range of former prosecutors struggled to point to any previous instance in which the Justice Department had abandoned its own case after obtaining a guilty plea. They portrayed the justification Shea pointed to -- that it would be difficult to prove to a jury beyond a reasonable doubt that the lies were material -- as dubious."A pardon would have been a lot more honest," said Samuel Buell, a former federal prosecutor who now teaches criminal law at Duke University.The law regarding what counts as "material" is extremely forgiving to the government, Buell added. The idea is that law enforcement is permitted to pursue possible theories of criminality and to interview people without having firmly established that there was a crime first.James G. McGovern, a defense lawyer at Hogan Lovells and a former federal prosecutor, said juries rarely bought a defendant's argument that a lie did not involve a material fact."If you are arguing 'materiality,' you usually lose, because there is a tacit admission that what you said was untrue, so you lose the jury," he said.No career prosecutors signed the motion. Shea is a former close aide to Barr. In January, Barr installed him as the top prosecutor in the district that encompasses the nation's capital after maneuvering out the Senate-confirmed former top prosecutor in that office, Jessie K. Liu.Soon after, in an extraordinary move, four prosecutors in the office abruptly quit the case against Trump's longtime friend Roger Stone. They did so after senior Justice Department officials intervened to recommend a more lenient prison term than standard sentencing guidelines called for in the crimes Stone was convicted of committing -- including witness intimidation and perjury -- to conceal Trump campaign interactions with WikiLeaks.It soon emerged that Barr had also appointed an outside prosecutor, Jeff Jensen, the U.S. attorney in St. Louis, to review the Flynn case files. The department then began turning over FBI documents showing internal deliberations about questioning Flynn, like what warnings to give -- even though such files are usually not provided to the defense.Flynn's defense team has mined such files for ammunition to portray the FBI as running amok in its decision to question Flynn in the first place. The questioning focused on his conversations during the transition after the 2016 election with the Russian ambassador about the Obama administration's imposition of sanctions on Russia for its interference in the American election.The FBI had already concluded that there was no evidence that Flynn, a former Trump campaign adviser, had personally conspired with Russia about the election, and it had decided to close out the counterintelligence investigation into him. Then questions arose about whether and why Flynn had lied to administration colleagues like Vice President Mike Pence about his conversations with the ambassador.Because the counterintelligence investigation was still open, the bureau used it as a basis to question Flynn about the conversations and decided not to warn him at its onset that it would be a crime to lie. Notes from Bill Priestap, then the head of the FBI's counterintelligence division, show that he wrote at one point about the planned interview: "What's our goal? Truth/admission or to get him to lie, so we can prosecute him or get him fired?"Barr has also appointed another outside prosecutor, John H. Durham, the U.S. attorney in Connecticut, to reinvestigate the Russia investigators even though the department's independent inspector general was already scrutinizing them.And his department has intervened in a range of other ways, from seeking more comfortable prison accommodations last year for Paul Manafort, Trump's former campaign chairman, to abruptly dropping charges in March against two Russian shell companies that were about to go to trial for financing schemes to interfere in the 2016 election using social media.Barr has let it be known that he does not think the FBI ever had an adequate legal basis to open its Russia investigation in the first place, contrary to the judgment of the Justice Department's inspector general.In an interview on CBS News on Thursday, Barr defended the dropping of the charges against Flynn on the grounds that the FBI "did not have a basis for a counterintelligence investigation against Flynn at that stage."Anne Milgram, a former federal prosecutor and former New Jersey attorney general who teaches criminal law at New York University, defended the FBI's decision to question Flynn in January 2017. She said that much was still a mystery about the Russian election interference operation at the time and that Flynn's lying to the vice president about his postelection interactions with a high-ranking Russian raised new questions.But, she argued, the more important frame for assessing the dropping of the case was to recognize how it fit into the larger pattern of the Barr-era department "undercutting the law enforcement officials and prosecutors who investigated the 2016 election and its aftermath," which she likened to "eating the Justice Department from the inside out."This article originally appeared in The New York Times.(C) 2020 The New York Times Company





x

Google employees are told to expect to work from home for the rest of the year, but a select few will be allowed to return to offices as soon as June

Google CEO Sundar Pichai has told employees to expect to work from home for the remainder of 2020, but will open offices for certain exceptions.





x

Ex-husband of Biden accuser Tara Reade said she told him of being sexual harassed: report

Biden has repeatedly denied Reade's allegation.





x

Boris Johnson, Carrie Symonds, and a baby in a very exclusive club

Baby Johnson joins Leo Blair and Florence Cameron to become a member of a very exclusive club.




x

Coronavirus: 'Virus not beaten' but UK can think about next phase, says Raab

The Foreign Secretary says the prime minister will outline what steps the UK "can responsibly take over the following weeks."




x

Coronavirus: Don't ban over-70s from lockdown easing, says ex-MP

Ann Clwyd argues against "blanket ban" on over-70s involvement in easing of virus restrictions.




x

Brexit: Simon Coveney says trade talks progress not good

The Irish deputy PM says time is short to reach a trade deal this year, with much still to do.




x

Met Police end probe into pro-Brexit campaigners

Leave campaigners were accused by the Electoral Commission of breaking the law over spending limits.




x

Coronavirus lockdown: UK 'should not expect big changes'

The PM will set out a "cautious" road map for the UK in his speech on Sunday, a cabinet minister says.




x

Coronavirus: Six money-saving ideas for lockdown and beyond

Millions of people are facing pay cuts or less work, so how can you make your money go further?




x

Extra substitutes among temporary law changes when leagues resume

Teams will be allowed to use five substitutes when the season resumes after a Fifa proposal to help with fixture congestion was approved.




x

Can robotaxis ease public transport fears in China?

More self-driving cabs are being launched in China at a time when people are worried about public transport.




x

Facebook and Google extend working from home to end of year

The tech giants plan to re-open offices soon but will allow staff to work remotely throughout 2020.




x

Xbox: Microsoft reveals first games for Series X console

The Xbox team shows off new footage of the highly anticipated Assassin's Creed: Valhalla.




x

RTX Voice: Noise-destroying tech put to the test

Two noise-cancelling AI systems - Nvidia RTX Voice and Krisp - are put to the test.




x

TileDB 2.0, Scylla 4.0, and CockroachDB raises extra funds

#303 — May 8, 2020

Read on the Web

Database Weekly

Introducing Scylla Open Source 4.0 — Scylla (a Cassandra-compatible NoSQL data store aiming to be the “world’s fastest column-store database”) now provides production-ready lightweight Transactions (LWT), a DynamoDB-compatible API (Alternator), and operator for Kubernetes, and more.

Dor Laor

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

????Live Coding: Guide to Grafana 101 - Getting Started with AlertsJoin us on May 20th to see how to use Grafana’s alerting functionality to get notified about anomalies in your data, dig into root causes, and respond to critical issues. Step-by-step demos + tips = cheaper, more flexible monitoring ✅.

Timescale sponsor

TileDB 2.0 and the Future of Data Science — TileDB is an embeddable storage engine focused on working with dense and sparse multi-dimensional arrays. It’s a C++ library with official Python, R, Java and Go integrations, but it can integrate with other database systems too. 2.0 introduces dataframe support, a new API for R, and support for Google Cloud Storage and Azure Blob Storage.

Stavros Papadopoulos

Time-Series Compression Algorithms, Explained — Delta-delta encoding, Simple-8b, XOR-based compression, and more - these algorithms aren’t magic, but combined they can save over 90% of storage costs and speed up queries. Here’s how they work.

Joshua Lockerman and Ajay Kulkarni

CockroachDB Creators Raise $87 Million of New Investment — Quite a raise and quite a valuation in these times for the creators of CockroachDB, a popular distributed SQL database.

Cockroach Labs

The Big Cloud Data Boom Gets Even Bigger, Thanks to COVID-19? — It’s not like the cloud was doing badly beforehand, but the pandemic is apparently encouraging companies to virtualize as much of their operations as possible.

Datanami

MongoDB Is Easy. Now Make It Powerful. Free Download for 30 Days. — Using MongoDB Atlas? Studio 3T is the professional GUI and IDE that unlocks the power you need.

Studio 3T sponsor

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using AWS API Gateway to Run Database Queries — API Gateway is commonly used to hook up HTTP endpoints to AWS Lambda functions but did you know it can directly connect to DynamoDB? (Or any AWS service that lets you query over the AWS API, so not RDS.)

Renato Byrro

How to Remain Agile with DynamoDB — Amazon DynamoDB delivers performance at scale but at a cost to flexibility (particularly early on in the development cycle when your eventual access patterns aren’t always known) – there are some mitigations, however.

Rob Cronin

Jobs

DevOps Engineer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Tooling

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

AvionDB: A Decentralised Database with MongoDB-like Developer Interface — An admittedloy ‘alpha stage’ database system built on top of OrbitDB, a serverless, peer-to-peer database that uses IFPS for storage and implements the core decentralized database logic/protocol.

Dappkit

mssql-cli, a CLI Tool to Manage SQL Server, Now on macOS and Linux — mssql-cli is a tool for working with SQL Server from the command line, complete with Intellisense, syntax highlighting, and paging.

Alan Yu (Microsoft)




x

Expressive Design Systems

Yesenia Perez-Cruz started her career as a designer at Happy Cog Philadelphia. From the first day, her design gifts were unmistakable. As her career progressed, she moved from one challenging role to another. At companies like Vox Media and Shopify, and at conferences around the world, she has been a design team leader, a popular […]

The post Expressive Design Systems appeared first on Zeldman on Web & Interaction Design.




x

A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX.

It’s one thing to seek diverse talent to add to your team, another to retain the people you’ve hired. Why do so many folks we bring in to add depth and breadth of experience to our design and business decision-making process end up leaving? Hear thoughtful, useful answers to this question and other mysteries of […]

The post A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX. appeared first on Zeldman on Web & Interaction Design.




x

Building Great User Experiences with Concurrent Mode and Suspense

At React Conf 2019 we announced an experimental release of React that supports Concurrent Mode and Suspense. In this post we’ll introduce best practices for using them that we’ve identified through the process of building the new facebook.com.

This post will be most relevant to people working on data fetching libraries for React.

It shows how to best integrate them with Concurrent Mode and Suspense. The patterns introduced here are based on Relay — our library for building data-driven UIs with GraphQL. However, the ideas in this post apply to other GraphQL clients as well as libraries using REST or other approaches.

This post is aimed at library authors. If you’re primarily an application developer, you might still find some interesting ideas here, but don’t feel like you have to read it in its entirety.

Talk Videos

If you prefer to watch videos, some of the ideas from this blog post have been referenced in several React Conf 2019 presentations:

This post presents a deeper dive on implementing a data fetching library with Suspense.

Putting User Experience First

The React team and community has long placed a deserved emphasis on developer experience: ensuring that React has good error messages, focusing on components as a way to reason locally about app behavior, crafting APIs that are predictable and encourage correct usage by design, etc. But we haven’t provided enough guidance on the best ways to achieve a great user experience in large apps.

For example, the React team has focused on framework performance and providing tools for developers to debug and tune application performance (e.g. React.memo). But we haven’t been as opinionated about the high-level patterns that make the difference between fast, fluid apps and slow, janky ones. We always want to ensure that React remains approachable to new users and supports a variety of use-cases — not every app has to be “blazing” fast. But as a community we can and should aim high. We should make it as easy as possible to build apps that start fast and stay fast, even as they grow in complexity, for users on varying devices and networks around the world.

Concurrent Mode and Suspense are experimental features that can help developers achieve this goal. We first introduced them at JSConf Iceland in 2018, intentionally sharing details very early to give the community time to digest the new concepts and to set the stage for subsequent changes. Since then we’ve completed related work, such as the new Context API and the introduction of Hooks, which are designed in part to help developers naturally write code that is more compatible with Concurrent Mode. But we didn’t want to implement these features and release them without validating that they work. So over the past year, the React, Relay, web infrastructure, and product teams at Facebook have all collaborated closely to build a new version of facebook.com that deeply integrates Concurrent Mode and Suspense to create an experience with a more fluid and app-like feel.

Thanks to this project, we’re more confident than ever that Concurrent Mode and Suspense can make it easier to deliver great, fast user experiences. But doing so requires rethinking how we approach loading code and data for our apps. Effectively all of the data-fetching on the new facebook.com is powered by Relay Hooks — new Hooks-based Relay APIs that integrate with Concurrent Mode and Suspense out of the box.

Relay Hooks — and GraphQL — won’t be for everyone, and that’s ok! Through our work on these APIs we’ve identified a set of more general patterns for using Suspense. Even if Relay isn’t the right fit for you, we think the key patterns we’ve introduced with Relay Hooks can be adapted to other frameworks.

Best Practices for Suspense

It’s tempting to focus only on the total startup time for an app — but it turns out that users’ perception of performance is determined by more than the absolute loading time. For example, when comparing two apps with the same absolute startup time, our research shows that users will generally perceive the one with fewer intermediate loading states and fewer layout changes as having loaded faster. Suspense is a powerful tool for carefully orchestrating an elegant loading sequence with a few, well-defined states that progressively reveal content. But improving perceived performance only goes so far — our apps still shouldn’t take forever to fetch all of their code, data, images, and other assets.

The traditional approach to loading data in React apps involves what we refer to as “fetch-on-render”. First we render a component with a spinner, then fetch data on mount (componentDidMount or useEffect), and finally update to render the resulting data. It’s certainly possible to use this pattern with Suspense: instead of initially rendering a placeholder itself, a component can “suspend” — indicate to React that it isn’t ready yet. This will tell React to find the nearest ancestor <Suspense fallback={<Placeholder/>}>, and render its fallback instead. If you watched earlier Suspense demos this example may feel familiar — it’s how we originally imagined using Suspense for data-fetching.

It turns out that this approach has some limitations. Consider a page that shows a social media post by a user, along with comments on that post. That might be structured as a <Post> component that renders both the post body and a <CommentList> to show the comments. Using the fetch-on-render approach described above to implement this could cause sequential round trips (sometimes referred to as a “waterfall”). First the data for the <Post> component would be fetched and then the data for <CommentList> would be fetched, increasing the time it takes to show the full page.

There’s also another often-overlooked downside to this approach. If <Post> eagerly requires (or imports) the <CommentList> component, our app will have to wait to show the post body while the code for the comments is downloading. We could lazily load <CommentList>, but then that would delay fetching comments data and increase the time to show the full page. How do we resolve this problem without compromising on the user experience?

Render As You Fetch

The fetch-on-render approach is widely used by React apps today and can certainly be used to create great apps. But can we do even better? Let’s step back and consider our goal.

In the above <Post> example, we’d ideally show the more important content — the post body — as early as possible, without negatively impacting the time to show the full page (including comments). Let’s consider the key constraints on any solution and look at how we can achieve them:

  • Showing the more important content (the post body) as early as possible means that we need to load the code and data for the view incrementally. We don’t want to block showing the post body on the code for <CommentList> being downloaded, for example.
  • At the same time we don’t want to increase the time to show the full page including comments. So we need to start loading the code and data for the comments as soon as possible, ideally in parallel with loading the post body.

This might sound difficult to achieve — but these constraints are actually incredibly helpful. They rule out a large number of approaches and spell out a solution for us. This brings us to the key patterns we’ve implemented in Relay Hooks, and that can be adapted to other data-fetching libraries. We’ll look at each one in turn and then see how they add up to achieve our goal of fast, delightful loading experiences:

  1. Parallel data and view trees
  2. Fetch in event handlers
  3. Load data incrementally
  4. Treat code like data

Parallel Data and View Trees

One of the most appealing things about the fetch-on-render pattern is that it colocates what data a component needs with how to render that data. This colocation is great — an example of how it makes sense to group code by concerns and not by technologies. All the issues we saw above were due to when we fetch data in this approach: upon rendering. We need to be able to fetch data before we’ve rendered the component. The only way to achieve that is by extracting the data dependencies into parallel data and view trees.

Here’s how that works in Relay Hooks. Continuing our example of a social media post with body and comments, here’s how we might define it with Relay Hooks:

// Post.js
function Post(props) {
  // Given a reference to some post - `props.post` - *what* data
  // do we need about that post?
  const postData = useFragment(graphql`
    fragment PostData on Post @refetchable(queryName: "PostQuery") {
      author
      title
      # ...  more fields ...
    }
  `, props.post);

  // Now that we have the data, how do we render it?
  return (
    <div>
      <h1>{postData.title}</h1>
      <h2>by {postData.author}</h2>
      {/* more fields  */}
    </div>
  );
}

Although the GraphQL is written within the component, Relay has a build step (Relay Compiler) that extracts these data-dependencies into separate files and aggregates the GraphQL for each view into a single query. So we get the benefit of colocating concerns, while at runtime having parallel data and view trees. Other frameworks could achieve a similar effect by allowing developers to define data-fetching logic in a sibling file (maybe Post.data.js), or perhaps integrate with a bundler to allow defining data dependencies with UI code and automatically extracting it, similar to Relay Compiler.

The key is that regardless of the technology we’re using to load our data — GraphQL, REST, etc — we can separate what data to load from how and when to actually load it. But once we do that, how and when do we fetch our data?

Fetch in Event Handlers

Imagine that we’re about to navigate from a list of a user’s posts to the page for a specific post. We’ll need to download the code for that page — Post.js — and also fetch its data.

Waiting until we render the component has problems as we saw above. The key is to start fetching code and data for a new view in the same event handler that triggers showing that view. We can either fetch the data within our router — if our router supports preloading data for routes — or in the click event on the link that triggered the navigation. It turns out that the React Router folks are already hard at work on building APIs to support preloading data for routes. But other routing frameworks can implement this idea too.

Conceptually, we want every route definition to include two things: what component to render and what data to preload, as a function of the route/url params. Here’s what such a route definition might look like. This example is loosely inspired by React Router’s route definitions and is primarily intended to demonstrate the concept, not a specific API:

// PostRoute.js (GraphQL version)

// Relay generated query for loading Post data
import PostQuery from './__generated__/PostQuery.graphql';

const PostRoute = {
  // a matching expression for which paths to handle
  path: '/post/:id',

  // what component to render for this route
  component: React.lazy(() => import('./Post')),

  // data to load for this route, as function of the route
  // parameters
  prepare: routeParams => {
    // Relay extracts queries from components, allowing us to reference
    // the data dependencies -- data tree -- from outside.
    const postData = preloadQuery(PostQuery, {
      postId: routeParams.id,
    });

    return { postData };
  },
};

export default PostRoute;

Given such a definition, a router can:

  • Match a URL to a route definition.
  • Call the prepare() function to start loading that route’s data. Note that prepare() is synchronous — we don’t wait for the data to be ready, since we want to start rendering more important parts of the view (like the post body) as quickly as possible.
  • Pass the preloaded data to the component. If the component is ready — the React.lazy dynamic import has completed — the component will render and try to access its data. If not, React.lazy will suspend until the code is ready.

This approach can be generalized to other data-fetching solutions. An app that uses REST might define a route like this:

// PostRoute.js (REST version)

// Manually written logic for loading the data for the component
import PostData from './Post.data';

const PostRoute = {
  // a matching expression for which paths to handle
  path: '/post/:id',

  // what component to render for this route
  component: React.lazy(() => import('./Post')),

  // data to load for this route, as function of the route
  // parameters
  prepare: routeParams => {
    const postData = preloadRestEndpoint(
      PostData.endpointUrl, 
      {
        postId: routeParams.id,
      },
    );
    return { postData };
  },
};

export default PostRoute;

This same approach can be employed not just for routing, but in other places where we show content lazily or based on user interaction. For example, a tab component could eagerly load the first tab’s code and data, and then use the same pattern as above to load the code and data for other tabs in the tab-change event handler. A component that displays a modal could preload the code and data for the modal in the click handler that triggers opening the modal, and so on.

Once we’ve implemented the ability to start loading code and data for a view independently, we have the option to go one step further. Consider a <Link to={path} /> component that links to a route. If the user hovers over that link, there’s a reasonable chance they’ll click it. And if they press the mouse down, there’s an even better chance that they’ll complete the click. If we can load code and data for a view after the user clicks, we can also start that work before they click, getting a head start on preparing the view.

Best of all, we can centralize that logic in a few key places — a router or core UI components — and get any performance benefits automatically throughout our app. Of course preloading isn’t always beneficial. It’s something an application would tune based on the user’s device or network speed to avoid eating up user’s data plans. But the pattern here makes it easier to centralize the implementation of preloading and the decision of whether to enable it or not.

Load Data Incrementally

The above patterns — parallel data/view trees and fetching in event handlers — let us start loading all the data for a view earlier. But we still want to be able to show more important parts of the view without waiting for all of our data. At Facebook we’ve implemented support for this in GraphQL and Relay in the form of some new GraphQL directives (annotations that affect how/when data is delivered, but not what data). These new directives, called @defer and @stream, allow us to retrieve data incrementally. For example, consider our <Post> component from above. We want to show the body without waiting for the comments to be ready. We can achieve this with @defer and <Suspense>:

// Post.js
function Post(props) {
  const postData = useFragment(graphql`
    fragment PostData on Post {
      author
      title

      # fetch data for the comments, but don't block on it being ready
      ...CommentList @defer
    }
  `, props.post);

  return (
    <div>
      <h1>{postData.title}</h1>
      <h2>by {postData.author}</h2>
      {/* @defer pairs naturally with <Suspense> to make the UI non-blocking too */}
      <Suspense fallback={<Spinner/>}>
        <CommentList post={postData} />
      </Suspense>
    </div>
  );
}

Here, our GraphQL server will stream back the results, first returning the author and title fields and then returning the comment data when it’s ready. We wrap <CommentList> in a <Suspense> boundary so that we can render the post body before <CommentList> and its data are ready. This same pattern can be applied to other frameworks as well. For example, apps that call a REST API might make parallel requests to fetch the body and comments data for a post to avoid blocking on all the data being ready.

Treat Code Like Data

But there’s one thing that’s still missing. We’ve shown how to preload data for a route — but what about code? The example above cheated a bit and used React.lazy. However, React.lazy is, as the name implies, lazy. It won’t start downloading code until the lazy component is actually rendered — it’s “fetch-on-render” for code!

To solve this, the React team is considering APIs that would allow bundle splitting and eager preloading for code as well. That would allow a user to pass some form of lazy component to a router, and for the router to trigger loading the code alongside its data as early as possible.

Putting It All Together

To recap, achieving a great loading experience means that we need to start loading code and data as early as possible, but without waiting for all of it to be ready. Parallel data and view trees allow us to load the data for a view in parallel with loading the view (code) itself. Fetching in an event handler means we can start loading data as early as possible, and even optimistically preload a view when we have enough confidence that a user will navigate to it. Loading data incrementally allows us to load important data earlier without delaying the fetching of less important data. And treating code as data — and preloading it with similar APIs — allows us to load it earlier too.

Using These Patterns

These patterns aren’t just ideas — we’ve implemented them in Relay Hooks and are using them in production throughout the new facebook.com (which is currently in beta testing). If you’re interested in using or learning more about these patterns, here are some resources:

  • The React Concurrent docs explore how to use Concurrent Mode and Suspense and go into more detail about many of these patterns. It’s a great resource to learn more about the APIs and use-cases they support.
  • The experimental release of Relay Hooks implements the patterns described here.
  • We’ve implemented two similar example apps that demonstrate these concepts:

    • The Relay Hooks example app uses GitHub’s public GraphQL API to implement a simple issue tracker app. It includes nested route support with code and data preloading. The code is fully commented — we encourage cloning the repo, running the app locally, and exploring how it works.
    • We also have a non-GraphQL version of the app that demonstrates how these concepts can be applied to other data-fetching libraries.

While the APIs around Concurrent Mode and Suspense are still experimental, we’re confident that the ideas in this post are proven by practice. However, we understand that Relay and GraphQL aren’t the right fit for everyone. That’s ok! We’re actively exploring how to generalize these patterns to approaches such as REST, and are exploring ideas for a more generic (ie non-GraphQL) API for composing a tree of data dependencies. In the meantime, we’re excited to see what new libraries will emerge that implement the patterns described in this post to make it easier to build great, fast user experiences.




x

Stimulus Reflex, and sending thanks to Matz

#498 — April 23, 2020

Read on the Web

Ruby Weekly

Credit: Divina Epiphania / Shutterstock.com

Mining for Malicious Ruby Gems: 700+ Gems Affected — Breathe easy as this was all resolved a month ago (and was too obscure to pay off for the hackers anyway) but a security research team recently found over 700 malicious Ruby gems that were subtle typos/adjustments of more popular gems (e.g. atlas-client vs atlas_client – could you tell which one is real?)

Tomislav Maljic

You Can Now Sponsor Matz on GitHub — I appreciate these are challenging times, but if you’ve ever wanted to give a big thank you to Matz, the creator of Ruby, here’s one way to do it. We’re sponsoring Matz now as without him, this newsletter wouldn’t exist! ???? Alternatively, if you have little to spare, maybe send him a thanks on Twitter?

GitHub Sponsors

Ruby Performance Tips — Here’s a collection of practical tips for improving Ruby performance for better user experiences, brought to you by Raygun. Read the tips here.

Raygun sponsor

Full Text Search in Milliseconds with Rails and Postgres — If you’ve never played with full text search with Postgres and Rails, this is a fine place to start. It covers LIKE/ILIKE, trigrams, and ‘proper’ full text searching. We also get to see how Leigh took a query from taking 130ms down to 7ms.

Leigh Halliday

▶  Introduction to Stimulus Reflex — Stimiulus Reflex makes SPA-type interactions very simple by using ActionCable to render pages and then diffing them on the client.

GoRails

Rails Performance: When is Caching the Right Choice? — Before you say “always”, understand that caching is not free and, if done incorrectly, can even make things worse.

Jonathan Miles

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Ruby Backend Developer (Austria) — We’re seeking mid-level and senior devs to join us and build top-class backend infrastructure for our adidas apps, used by millions. Our stack includes: jRuby, Sinatra, Sidekiq, MySQL, & MongoDB.

Runtastic

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

How to Customize Webpack in Rails Apps — How to go about configuring webpack when tweaking webpacker.yml just isn’t enough.

Ross Kaffenberger

RSpec Given/When/Then with Symbols — An interesting, alternative way to structure a BDD feature in RSpec. I think I prefer the underscores but YMMV.

Caius Durling

Looking Inside a Ruby Gem — Piotr decomposes a .gem file which turns out to just be a collection of gzipped and tarred files, only some of which are the code.

Piotr Murach

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, you will learn about the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Passing Rails Controller Params to SidekiqActionController::Parameters can give Sidekiq issues.

Prathamesh Sonpatki

Catchup Subscriptions with Rails Event Store

Miroslaw Praglowski

Logic-less Ruby Templates with Mustache

David Santangelo

▶  Discussing Ruby for Good with Sean Marcia — Sean talks about founding Ruby For Good (an event about philanthropic Ruby development) and some of the projects it has been responsible for creating.

Ruby Rogues podcast

???? Code and Tools

Impressionist 2.0: A Plugin to Log Impressions in Rails Apps — Impressionist tracks page views and impressions. v2.0 has just dropped but they’re also are looking for new maintainers, so contact them if you want to get involved.

Charlotte Ruby Group

acli 0.3: A Command Line Client for Action Cable — Interesting on two fronts.. first, because it’s an mruby app, and we don’t see many of those, and second, because it lets you play with Action Cable channels in any easier way.

Vladimir Dementyev

Undercover: A Tool to Stop You Shipping Untested Code — It’s like RuboCop but for code coverage rather than code style.

Jan Grodowski

How to Monitor Your Host Metrics Automatically

AppSignal sponsor

Bridgetown: A Modern Ruby (JAMstack) Web Framework — Bridgetown is a new Ruby-based static-site generator based on a fork of Jekyll. It supports plugins and Webpack, so you can use your front-end framework of choice.

Bridgetown

net-ssh 6.0: A Pure Ruby Implementation of the SSH2 Client Protocol — Yes, you can write programs that invoke and interact with processes on remote servers, via SSH2, all in Ruby.

Buck, Fazekas, et al.




x

An interview with Ruby ETL expert Thibaut Barrère

#499 — April 30, 2020

Read on the Web

???? Occasionally we run interviews in Ruby Weekly and we're back with another one.. with long time Rubyist and Kiba maintainer, Thibaut Barrère. Be sure to check out the bottom of this issue to read it, especially if you ever run ETL jobs with Ruby! ????

Ruby Weekly

▶  Let's Build a Twitter Clone in 10 Minutes with Rails, CableReady, and StimulusReflex — You know that cloning Twitter in 10 minutes is impossible, but what about the core mechanism of the idea? After a slow first minute, this video does a pretty good job of showing off some techniques you might not have used before.

Nate Hopkins

discuss.rubyonrails: The Rails Project Discussion Forum — Basically a Web version of the Rails mailing lists and a worthwhile place to head if you want to suggest features, ask questions, etc.

Ruby on Rails Discussions

Easy Rails Deployments — Deploy your Ruby, Rails, Sinatra, and Rack application to any cloud or server. Cloud 66 offers a scalable Heroku alternative that doesn't lock you in. Try it free and get extra $66 free credits with the code: 'Ruby-Weekly'.

Cloud 66 sponsor

sequel-activerecord-adapter: Allows Sequel to Reuse an ActiveRecord Connection — If you want to use more Sequel or migrate in that direction, this makes it easy.

Janko Marohnić

Ruby Adds Experimental Support for 'End-Less' Method Definitions — We’ve touched on this in a previous issue, but if you fancy a proper blog post with examples, this is more accessible than digging through feature tracker discussions. I’m not a fan of this syntax myself yet, but Prateek does a good job of selling it.

Prateek Choudhary

CableReady: Trigger Client-Side DOM Changes from Server-Side Ruby — If you skipped the video above because it’s a video (and I know many of you do ????) CableReady is still worth checking out. It aims to “complete the ActionCable story” by providing a way to directly interact with clients over ActionCable WebSockets. The docs will help you get the idea.

Hopsoft

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Security Engineer (Remote) — Are you an engineer with experience in Rails and/or Go? Join our team and help secure our apps and cloud infrastructure.

Shogun

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

A Practical Use for PStorePStore is one of the older parts of Ruby’s stdlib and lets you persist (and restore) a Ruby hash to disk. I haven’t seen it in Ruby code for years but Arkency have found a neat, modern use case.

Paweł Pacana

▶  Drag and Drop Sortable Lists with Rails and Stimulus JS — How to wire up drag and drop lists with a Rails app using Sortable, Stimulus and acts_as_list.

Go Rails

Setting Up Multi-Factor Authentication for RubyGems.org — If you’re a registered user of the official Ruby Gems repository, you should have had an email this week about securing your account using 2FA. This is just my own reminder that this is a good idea especially if you publicly publish widely used gems! :-)

Rubygems.org

Let’s Explore Big-O Notation With Ruby ???? — Learn to use Big-O to look at an algorithm and easily discern its efficiency, without having to run a profiling tool.

Honeybadger sponsor

▶  Discussing Docker and Kubernetes with Kelsey Hightower — A worthwhile show to listen to if Kubernetes and Docker intimidate you but you want to know a little more. Kelsey is good at breaking these things down into understandable pieces.

Rails with Jason Podcast podcast

Rails System Tests in Docker — We’re seeing an uptick in articles about system tests in Rails. Here’s how to integrate them into your development Docker setup.

Hint.io

The Difference Between System Specs and Feature Specs — If you’ve felt the difference between RSpec’s “feature specs” and “system specs” is quite subtle, this explanation will help.

Jason Swett

Why Rubyists Should Consider Learning Go — If you want compilation and a type system, Crystal is probably a better fit for Rubyists, but Go is undoubtedly a neat language and ecosystem (and if you do end up in the Go world, check out our Go weekly! ????)

Ayooluwa Isaiah

Building a Ruby CLI with Thor

Daniel Gómez

???? Code and Tools

git curate: Peruse and Delete git Branches Ergonomically — Got a repo cluttered with branches here and there? git curate aims to cure the pains of getting those branches back under control.

Matt Harvey

MessageBus: A Reliable and Robust Messaging Bus for Ruby and Rack

Sam Saffron

Are You Spending Too Much on Heroku?

Rails Autoscale sponsor

bootstrap_form: A Rails Form Builder for Bootstrap v4-Style Forms

Bootstrap Ruby

ActiveModelAttributes: The Active Record Attributes API, but for Active Model — Brings some of the goodies of the Rails 5 Active Record attributes API to ActiveModel too. 1.6.0 just dropped.

Karol Galanciak

???? A Q&A with…
Thibaut Barrère
Creator of Kiba ETL

Thibaut Barrère is a long-time Rubyist and data engineer who built and maintains the popular Extract, Transform, and Load (ETL) framework Kiba. We asked him some questions about his work:

What inspired you to create Kiba?

A lot of my work since ~2005 has been focused on data integration (making systems speak together), data aggregation etc. I sometimes used GUI-based tools like Microsoft SSIS which, while powerful, are quite far from the coding experience.

I was already using Ruby at that time, and was happy to discover activewarehouse-etl (maintained by Anthony Eden, who runs DNSimple now) providing a Ruby DSL to declare data pipelines. I used it for a while with very good success to implement data extractions and business intelligence ETL, and ultimately took over the maintenance.

In the long run, though (as I explained in a recent Paris.rb talk), the balance between the the cost of OSS maintenance and the usability for my billable and non-billable use-cases proved to be not good enough, which made me decide to stop the maintenance, sadly.

I still wanted to use Ruby to write data pipelines, but I needed to cover more use-cases and reduce the OSS maintenance burden at the same time. This ultimately led me to write and share (in 2015) Kiba ETL, a focused DSL for declarative processing, matching those criterias.

Do you find yourself adding more features while in quarantine?

Before the quarantine, I directed my Kiba bandwith and focus on finalizing Kiba v3 and rewriting the documentation from scratch, to properly encourage best practices I’ve been discovering. I also created experimental branches for Ruby 2.7/2.8 keywords.

During the quarantine, I've reduced client work and OSS work too, to focus on shipping Kiba Pro v2 (which I’ll announce shortly officially). I’ve extracted and generalized (from real-life projects) very useful components, such as a “batch SQL lookup” (useful when replacing relationships keys during data migrations and datawarehouses sync code, in batch rather than row by row), a “file lock” to ensure a single job runs at once, and a “parallel transform” to achieve easy concurrency for things like HTTP queries.

What's the wildest ETL that you've encountered?

Getting the data out of a system which is actively not acting in that direction is always a bit wild.. One can see all types of fancy stuff on the field. For instance, it is not uncommon to have an ETL process start a headless browser, jump through pages, just to get to the CSV/PDF/Excel file that you will then use as your data source!

You can also end up having to figure out ways to read or write very old file formats at times. Recently I wrote a Kiba component to generate a COBOL delimited file, for instance. In large companies, a very widely used I/O is good old SFTP, far away from modern APIs and formats.

Can you tell us how to say your last name? ????

I had to deploy a page to my blog to answer that question properly ????. You’ll find out how to say my name here.

Merci Thibaut!

You can read some of Thibaut's posts on his blog and find out more about Kiba ETL here.