95 lines
3.2 KiB
Markdown
95 lines
3.2 KiB
Markdown
# Debugging Firefox with Pernosco
|
||
|
||
This page is intended to help Firefox/Gecko developers get started using [Pernosco](https://pernos.co/)
|
||
to debug Firefox.
|
||
|
||
Mozilla employees have a paid subscription to Pernosco, and are encouraged to use it.
|
||
Contributors can use Pernosco as well. [A free account](https://pernos.co/pricing/)
|
||
comes with 5 uploads to see if it is suitable for your workflow.
|
||
|
||
## Prerequisites
|
||
|
||
- You need the ability to produce an {code}`rr` trace. See {doc}`debugging_firefox_with_rr`
|
||
- Employees, you can also generate traces for Pernosco for some CI job failures via
|
||
Treeherder.
|
||
|
||
## Workflow
|
||
|
||
At a high level, working with Pernosco consists of recording a trace then uploading
|
||
the trace for processing.
|
||
|
||
### Install your Pernosco configuration
|
||
|
||
If you are using your own Pernoso account, follow the instructions
|
||
[here](https://pernos.co/account) to install your credentials.
|
||
|
||
Mozilla employees, see the [Pernosco install and configuration doc](https://docs.google.com/document/d/1cb-mkNyJYMudpS0a-R6zxy0kOMIRUSKrxxL-E9zFGKE/edit)
|
||
for credentials.
|
||
|
||
In general, you will want:
|
||
|
||
- Your email in {code}`~/.config/pernosco/user`
|
||
- Your group in {code}`~/.config/pernosco/group`
|
||
- Your key in {code}`~/.config/pernosco/user_secret_key`
|
||
|
||
### Gather a Local Trace
|
||
|
||
In general, to gather a trace follow the instruction for recording {code}`rr` traces
|
||
in {doc}`debugging_firefox_with_rr`. However, to gather a local rr trace that’s
|
||
compatible with Pernosco you need to disable some incompatible CPU features (currently
|
||
only AVX-512)
|
||
|
||
On an Intel machine, you can disable this at the command line. To simplify things a
|
||
shell script {code}`pernosco-record` can be created like so,
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
|
||
rr record --disable-avx-512 "$@"
|
||
```
|
||
|
||
replacing plain {code}`rr` in command lines ({code}`mach --debugger pernosco-record`)
|
||
|
||
On AMD Zen Machines, you can disable AVX-512 and 5 level page tables at boot.
|
||
|
||
On a Debian derived distro this involves editing {code}`/etc/default/grub` and
|
||
adding {code}`clearcpuid=304,no5lvl` to {code}`GRUB_CMDLINE_LINUX_DEFAULT`, then
|
||
running {code}`sudo update-grub` after.
|
||
|
||
### Upload the trace
|
||
|
||
Find the trace you’re interested in {code}`~/.local/share/rr/<trace>`, and call {code}`pernosco-submit`
|
||
|
||
```bash
|
||
pernosco-submit upload ~/.local/share/rr/<trace> PATH_TO_CENTRAL --title "Name Of Trace"
|
||
```
|
||
|
||
where {code}`PATH TO CENTRAL` is the path to the source of your build, checked out
|
||
to the revision matching the build.
|
||
|
||
The last trace recorded is always symlinked, so you can use {code}`~/.local/share/rr/latest-trace` too.
|
||
|
||
### Wait for email
|
||
|
||
After the trace is processed, you'll get an email with a link to your trace. Good
|
||
hunting!
|
||
|
||
## Treeherder Workflow
|
||
|
||
You can also try to get traces generated via the Treeherder interface. On a failing
|
||
test that is eligible, in "Artifacts and Debugging Tools", you'll see a link to
|
||
|
||
```{image} ../img/record-this-failure.png
|
||
:width: 35 %
|
||
```
|
||
|
||
This link brings you to the self-service reproducer:
|
||
|
||
```{image} ../img/self-service.png
|
||
:width: 60 %
|
||
```
|
||
|
||
Select a test and a count, and whether or not chaos-mode should be used, then click
|
||
reproduce. At this point there is a **long** wait; the page will notify you when
|
||
reproduction has started with an {code}`alert` dialog.
|