💾 Archived View for aphrack.org › issues › phrack66 › 13.gmi captured on 2021-12-03 at 14:04:38. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
==Phrack Inc.==
Volume 0x0d, Issue 0x42, Phile #0x0D of 0x11
|=----------------------------------------------------------------------=|
|=---------=[ Hacking the Cell Broadband Engine Architecture ]=---------=|
|=-------------------=[ SPE software exploitation ]=--------------------=|
|=----------------------------------------------------------------------=|
|=--------------=[ By BSDaemon ]=----------=|
|=--------------=[ <bsdaemon *noSPAM* risesecurity_org> ]=----------=|
|=----------------------------------------------------------------------=|
"There are two ways of
constructing a software design.
One way is to make it so simple
that there are obviously no
deficiencies. And the other way
is to make it so complicated that
there are no obvious deficiencies"
- C.A.R. Hoare
------[ Index
1 - Introduction
1.1 - Paper structure
2 - Cell Broadband Engine Architecture
2.1 - What is Cell
2.2 - Cell History
2.2.1 - Problems it solves
2.2.2 - Basic Design Concept
2.2.3 - Architecture Components
2.2.4 - Processor Components
2.3 - Debugging Cell
2.3.1 - Linux on Cell
2.3.2 - Extensions to Linux
2.3.2.1 - User-mode
2.3.2.2 - Kernel-mode
2.3.3 - Debugging the SPE
2.4 - Software Development for Linux on Cell
2.4.1 - PPE/SPE hello world
2.4.2 - Standard Library Calls from SPE
2.4.3 - Communication Mechanisms
2.4.4 - Memory Flow Control (MFC) Commands
2.4.5 - Direct Memory Access (DMA) Commands
2.4.5.1 - Get/Put Commands
2.4.5.2 - Resources
2.4.5.3 - SPE 2 SPE Communication
3 - Exploiting Software Vulnerabilities on Cell SPE
3.1 - Memory Overflows
3.1.1 - SPE memory layout
3.1.2 - SPE assembly basics
3.1.2.1 - Registers
3.1.2.2 - Local Storage Addressing Mode
3.1.2.3 - External Devices
3.1.2.4 - Instruction Set
3.1.3 - Exploiting software vulnerabilities in SPE
3.1.3.1 - Avoiding Null Bytes
3.1.4 - Finding software vulnerabilities on SPE
4 - Future and other uses
5 - Acknowledgements
6 - References
7 - Notes on SDK/Simulator Environment
8 - Sources
------[ 1 - Introduction
This article is all about Cell Broadband Architecture Engine [1], a new
hardware designed by a joint between Sony [2], Toshiba [3] and IBM [4].
As so, lots of architecture details will be explained, and also many
development differences for this platform.
The biggest differentiator between this article and others released about
this subject, is the focus on the architecture exploitation and not the
use of the powerful processor resources to break code [5] and of course,
the focus in the differentiators of the architecture, which means the SPU
(synergestic processor unit) and not in the core (PPU - power processor
unit) [6], since the core is a small-modified power processor (which
means, all shellcodes for Linux on Power will also works for the core and
there is just small differences in the code allocation and stuffs like
that).
It's important to mention that everything about Cell tries to focus in the
Playstation3 hardware, since it's cheap and widely deployed, but there is
also big machines made with this processor [7], including the #1 in the
list of supercomputers [8].
---[ 1.1 - Paper structure
The idea of this paper is to complete the studies about Cell, putting all
the information needed to do security research, focused in software
exploitation for this architecture together.
For that, the paper have been structured in two important portions:
Chapter 2 will be all about the Cell Architecture and how to develop for
this architecture. It includes many samples and explains the
modifications done to Linux in order to get the best from this
architecture. Also, it gives the knowledge needed in order to go further
in software exploitation for this arch. Chapter 3 is focused in the
exploitation of the SPU processor, showing the simple memory layout it has
and how to write a shellcode for the purpose of gaining control over an
application running inside the SPU.
------[ 2 - Cell Broadband Engine Architecture
From the IBM Research [9]: "The Cell Architecture grew from a challenge
posed by Sony and Toshiba to provide power-efficient and cost-effective
high-performance processing for a wide range of applications, including
the most demanding consumer appliance: game consoles. Cell - also known as
the Cell Broadband Engine Architecture (CBEA) - is an innovative solution
whose design was based on the analysis of a broad range of workloads in
areas such as cryptography, graphics transform and lighting, physics,
fast-Fourier transforms (FFT), matrix operations, and scientific
workloads. As an example of innovation that ensures the clients' success,
a team from IBM Research joined forces with teams from IBM Systems
Technology Group, Sony and Toshiba, to lead the development of a novel
architecture that represents a breakthrough in performance for consumer
applications. IBM Research participated throughout the entire development
of the architecture, its implementation and its software enablement,
ensuring the timely and efficient application of novel ideas and
technology into a product that solves real challenges."
It's impossible to not get excited with this. A so 'powerful' and
versatile architecture, completely different from what we usually seen is
an amazing stuff to research for software vulnerabilities. Also, since
it's supposed to be widely deployed, there will be an infinite number of
new vulnerabilities coming on in the near future. I wanted to exploit
those vulnerabilities.
---[ 2.1 - What is Cell
As must be already clear to the reader, I'm not talking about phones here.
Cell is a new architecture, which cames to solve some of the actual
problems in the computer industry.
It's compatible with a well-known architecture, which are the Power
Architecture, keeping most of it's advantages and solving most of it's
problems (if you cannot wait until know what problems, go to 2.2.1
section).
---[ 2.2 - Cell History
The focus of this section is just to give a timeline vision for the
reader, not been detailed at all.
The architecture was born from a joint between IBM, Sony and Toshiba,
formed in 2000.
They opened a design center in March 2001, based in Austin, Texas (USA).
In the spring of 2004, a single Cell BE became operational. In the summer
of the same year, a 2-way SMP version was released.
The first technical disclosures came just in February 2005, with the
simulator [10] and open-source SDK [11] (more on that later) been released
in November of the same year. In the same month, Mercury started to sell
Cell (yeah, sell Cell sounds funny) machines.
Cell Blades was announced by IBM in February of 2006. The SDK 1.1 was
released in July of the same year, with many improvements. The latest
version is 3.1.
---[ 2.2.1 - Problems it solves
The computer technology have been evolving along the years, but always
suffering and trying to avoid some barriers.
Those barriers are physically impossible to be bypassed and that's why the
processor clock stopped to grow and multi-core architectures been focused.
Basically we have three big walls (barriers) to the speedy grow:
- Power wall
It's related to the CMOS technology limits and the hard limit to
the acceptable system power
- Memory wall
Many comparisons and improvements trying to avoid the DRAM latency
when compared to the processor frequency
- Frequency wall
Diminishing return from deeper pipelines
For a new architecture to work and be widely deployed, it was also
important to keep the investments in software development.
Cell accomplish that being compatible with the 64 bits Power Architecture,
and attacks the walls in the following ways:
- Non-homogeneous coherent multi-processor and high design
frequency at a low operating voltage with advanced power
management attacks the 'power wall'.
- Streaming DMA architecture and three-level memory model (main
storage, local storage and register files) attacks the 'memory
wall'.
- Non-homogeneous coherent multi-processor, highly-optimized
implementation and large shared register files with software controlled
branching to allow deeper pipelines attacks the 'frequency wall'.
It have been developed to support any OS, which means it supports
real-time operating system as well non-real time operating systems.
---[ 2.2.2 - Basic Design Concept
The basic concept behind cell is it's asymmetric multi-core design. That
permits a powerful design, but of course requires specific-developed
applications to achieve the most of the architecture.
Knowing that, becomes clear that the understanding of the new component,
which is called SPU (synergistic processor unit) or SPE (synergistic
processor element) proofs to be essential - see the next section for a
better understanding of the differences between SPU and SPE.
---[ 2.2.3 - Architecture Components
In cell what we have is a core processor, called Power Processor Element
(PPE) which control tasks and synergistic processor elements (SPEs) for
data-intensive processing.
The SPE consists of the synergistic processor unit (SPU), which are a
processor itself and the memory flow control (MFC), responsible for the
data movements and synchronization, as well for the interface with the
high-performance element interconnect bus (EIB).
Communications with the EIB are done in a 16B/cycle, which means that each
SPU is interconnected at that speedy with the bus, which supports
96B/cycle.
Refer to the picture architecture-components.jpg in the directory images
of the attached file for a visual of the above explanation.
---[ 2.2.4 - Processor Components
As said, the Power Processor Element (PPE) is the core processor which
control tasks (scheduling). It is a general purpose 64 bit RISC processor
(Power architecture).
It's 2-way hardware multithreaded, with a L1: 32KB I and D caches and L2:
512KB cache.
Has support for real-time operations, like locking the L2 cache and the
TLB (also it supports managed TLB by hardware and software). It has
bandwidth and resource reservation and mediated interrupts.
It's also connected to the EIB using a 16B/cycle channel (figure
processor-components.jpg).
The EIB itself supports four 16 bytes data rings with simultaneous
transfers per ring (it will be clarified later).
This bus supports over 100 simultaneous transactions achieving in each bus
data port more than 25.6 Gbytes/sec in each direction.
On the other side, the synergistic processor element is a simple RISC
user-mode architecture supporting dual-issue VMX-like, graphics SP-float
and IEEE DP-float.
Important to note that the SPE itself has dedicated resources: unified 128
x 128 bit register files and 256KB local storage. Each SPE has a
dedicated DMA engine, supporting 16 requests.
The memory management on this architecture simplified it's use, with the
local storage of the SPE being aliased into the PPE system memory (figure
processor-components2.jpg).
MFC in the SPE acts as the MMU providing controls over the SPE DMA access
and it's compatible with the PowerPC Virtual Memory layout and is software
controllable using PPE MMIO.
DMA access supports 1,2,4,8...n*16 bytes transfer, with a maximum of 16 KB
for I/O, and with two different queues for DMA commands: Proxy & SPU
(more on this later).
EIB is also connected in a broadband interface controller (BIC). The
purpose of this controller is to provide external connectivity for
devices. It supports two configurable interfaces (60 GB/s) with a
configurable number of bytes, coherent (BIF) and/or I/O (IOIFx) protocols,
using two virtual channels per interface, and multiple system
configurations.
The memory interface controller (MIC) is also connected to the EIB and is
a Dual XDR controller (25.6 GB/s) with ECC and suspended DRAM support
(figure processor-components3.jpg).
Still are missing two more components: The internal interrupt controller
(IIC) and the I/O Bus Master Translation (IOT) (figure
processor-components4.jpg).
The IIC handles the SPE interrupts as well as the external interrupts and
interrupts comming from the coherent interconnect and the IOIF0 and IOIF1.
It is also responsible for the interrupt priority level control and for
the interrupt generation ports for IPI. Note that the IIC is duplicated
for each PPE hardware thread.
IOT translates bus addresses to system real addresses, supporting two
level translations:
- I/O segments (256 MB)
- I/O pages (4K, 64K, 1M, 16M bytes)
Interesting is the resource of I/O device identifier per page for LPAR use
(blades) and IOST/IOPT caches managed by software and hardware.
---[ 2.3 - Debugging Cell
As the bus is a high-speedy circuit, it's really difficult to debug the
architecture and better seen what is going on.
For that, and also to made it easy to develop software for Cell, IBM
Research developed a Cell simulator [10] in which you may run Linux and
install the software development kit [11].
The IBM Linux Technology Center brazilian team developed a plugin for
eclipse as an IDE for the debugger and SDK. Putting it all together is
possible to have the toolkit installed in a Linux machine, running the
frontends for the simulator and for the SDK. The debugging interface is
much better using this frontends. Anyway, it's important to notice that
it's just a frontend for the normal and well know linux tools with
extended support to Cell processor (GDB and GCC).
---[ 2.3.1 - Linux on Cell
Linux on cell is an open-source git branch and is provided in the PowerPC
64 kernel line.
It started in the 2.6.15 and is evolving to support many new features,
like the scheduling improvements for the SPUs (actually it can be
preempted, and my big friend Andre Detsch who reviewed this article was
one of the biggest contributors to create an stable code here).
On Linux it added heterogeneous lwp/thread model, with a new SPE thread
model (really similar to the pthreads library as we will see later),
supporting user-mode direct and indirect SPE access, full-preemptive SPE
context management and for that, spe_ptrace() was create and it's support
added to GDB, spe_schedule() for thread to physical spe assigment (it is
not anymore FIFO - run until completion).
As a note, the SPE threads shares it's address space with the parent PPE
process (using DMA), demand paging for SPE access and shared hardware page
table with PPE.
An implementation detail is the PPE proxy thread allocated for each SPE to
provide a single namespace for both PPE and SPE and assist in SPE
initiated C99 and Posix library services.
All the events, error and signal handling for SPEs are done by the parent
PPE thread.
The ELF objects for SPE are wrapped into PPE objects with an extended GLD.
---[ 2.3.2 - Extensions to Linux
Here I'll try to provide some details for Linux running under a Cell
Hardware. The base hardware used for this reference is a Playstation 3,
which has 8 SPUs, but one is reserved with the purpose of redundancy and
another one is used as hypervisor for a custom OS (in this case, Linux).
All the details are valid for any Linux on Cell and we will provide an
top-down view approach.
---[ 2.3.2.1 - User-mode
Cell supports both power 32 and 64 bits applications, as well as 32 and 64
cell workloads. It has different programming modes, like RPC, devices
subsystems and direct/indirect access.
As already said, it has heterogeneous threads: single SPU, SPU groups and
shared memory support.
It runs over a SPE management runtime library, with 32 and 64 bits. This
library interacts with the SPUFS filesystem (/spu/thread#/) in the
following ways:
* Open, close, read, write the files:
- mem
This file provides access to the local storage
- regs
Access to the 128 register of 128 bits each
- mbox
spe to ppe mailbox
- liox
spe to ppe interrupt mailbox
- xbox_stat
Get the mailbox status
- signal1
Signal notification acess
- signal2
Signal notification acess
- signalx_type
Signal type
- npc
Read/write SPE next program counter (for debugging)
- fpcr
SPE floating point control/status register
- decr
SPE decrementer
- decr_status
SPE decrementer status
- spu_tag_mask
Access tag query mask
- event_mask
Access spe event mask
- srr0
Access spe state restore register 0
* open, close mmap the files:
- mem
Program State access of the Local Storage
- signal1
Direct application access to signal 1
- signal2
Direct application access to signal 2
- cntl
Direct application access to SPE controls, DMA queues and
mailboxes
The library also provides SPE task control system calls (to interact with
the SPE system calls implemented in kernel-mode), which are:
- sys_spu_create_thread
Allocates a SPE task/context and creates a directory in SPUFS
- sys_spu_run
Activates a SPU task/context on a physical SPE and
blocks in the kernel as a proxy thread to handle the events
already mentioned
Some functions provided by the library are related to the management of
the spe tasks, like spe create group, create thread, get/set affinity,
get/set context, get event, get group, get ls, get ps area, get threads,
get/set priority, get policy, set group defaults, group max, kill/wait,
open/close image, write signal, read in_mbox, write out_mbox, read mbox
status.
Obviously the standard 32 and 64 bits powerpc ELF (binary) interpreters,
it is provided a SPE object loader, responsible for understand the
extension to the normal objects already mentioned and for initiate the
loading of the SPE threads.
Going down, we have the glibc and other GNU libraries, both supporting 32
and 64 bits.
---[ 2.3.2.2 - Kernel-mode
The next layer is the normal system-call interface, where we have the SPU
management framework (through special files in the spufs) and
modifications in the exec* interface, in a 64bit kernel.
This modification is done through a special misc format binary, called SPU
object loader extension.
Of course there is other kernel extensions, the SPUFS filesystem, which
provides the management interface and the SPU allocation, scheduling and
dispatch.
Also, we do have the Cell BE architecture specific code, supporting multi
and large pages, SPE event & fault handling, IIC and IOMMU.
Everything is controlled by a hypervisor, since Linux is what is called a
custom OS when running in a Playstation3 hardware (the hypervisor is
responsible for the protection of the 'secret key' of the hardware and
knowing how to exploit SPU vulnerabilities plus some fuzzing on the
hypervisor may be the needed knowledge to break the game protection copy
in this hardware).
---[ 2.3.3 - Debugging the SPE
The SDK for Linux on Cell provides good resources for Debugging and better
understanding of what is going on.
It's important to note the environment variables that control the
behaviour of the system.
So, if you set the SPU_INFO, for example, the spe runtime library will
print messages when loading a SPE ELF executable (see above).
---------- begin output ----------
# export SPU_INFO=1
# ./test
Loading SPE program: ./test
SPU LS Entry Addr : XXX
---------- end output ----------
And it will also print messages before starting up a new SPE thread, like:
---------- begin output ----------
Starting SPE thread 0x..., to attach debugger use: spu-gdb -p XXX
---------- end output ----------
When planning to use the spu-gdb to debug a SPU thread, it's important to
remember the SPU_DEBUG_START environment variable, which will include
everything provided by the SPU_INFO and will stop the thread until a
debugger is attached or a signal is received.
Since each SPU register can hold multiple fixed (or floating) point values
of different sizes, for GDB is provided a data structure that can be
accessed with different formats. So, specifying the field in the data
structure, we can update it using different sizes as well:
---------- begin output ----------
(gdb) ptype $r70
type = union __gdb_builtin_type_vec128 {
int128_t uint128;
float v4_float[4];
int32_t v4_int32[4];
int16_t v8_int16[8];
int8_t v16_int8[16];
}
(gdb) p $r70.uint128
$1 = 0x00018ff000018ff000018ff000018ff0
(gdb) set $r70.v4_int[2]=0xdeadbeef
(gdb) p $r70.uint128
$2 = 0x00018ff000018ff0deadbeef00018ff0
---------- end output ----------
To permit you to better understand when the SPU code starts the execution
and follow it gdb also included an interesting option:
---------- begin output ----------
(gdb) set spu stop-on-load
(gdb) run
...
(gdb) info registers
---------- end output ----------
Another important information for debugging your code is to understand the
internal sizes and be prepared for overlapping. Useful information can
be get using the following fragment code inside your spu program (careful:
It's not freeing the allocated memory).
--- code ---
extern int _etext;
extern int _edata;
extern int _end;
void meminfo(void)
{
printf("\n&_etext: %p", &_etext);
printf("\n&_edata: %p", &_edata);
printf("\n&_end: %p", &_end);
printf("\nsbrk(0): %p", sbrk(0));
printf("\nmalloc(1024): %p", malloc(1024));
printf("\nsbrk(0): %p", sbrk(0));
}
--- end code ---
And of course you can also play with the GCC and LD arguments to have more
debugging info:
--- code ---
# vi Makefile
CFLAGS += -g
LDFLAGS += -Wl,-Map,map_filename.map
--- end code ---
---[ 2.4 - Software Development for Linux on Cell
In this chapter I will introduce the inners of the Cell development,
giving the basic knowledge necessary to better understand the further
chapters.
---[ 2.4.1 - PPE/SPE hello world
Every program in Cell that uses the SPEs needs to have at least two source
codes. One for the PPE and another one for the SPE.
Following is a simple code to run on the SPE (it's also in the attached
tar file :
--- code ---
#include <stdio.h>
int main(unsigned long long speid, unsigned long long argp, unsigned long long envp)
{
printf("\nHello World!\n");
return 0;
}
--- end code ---
The Makefile for this code will look like:
--- code ---
PROGRAM_spu = hello_spu
LIBRARY_embed = hello_spu.a
IMPORTS = $(SDKLIB_spu)/libc.a
include ($TOP)/make.footer
--- end code ---
Of course it looks like any normal code. The PPE as already explained is
the responsible for the creation of the new thread and allocation in the
SPE:
--- code ---
#include <stdio.h>
#include <libspe.h>
extern spe_program_handle_t hello_spu;
int main(void)
{
int speid, status;
speid=spe_create_thread(0, &hello_spu, NULL, NULL, -1, 0);
spe_wait(speid, &status, 1);
return 0;
}
--- end code ---
With the following Makefile:
--- code ---
DIRS = spu
PROGRAM_ppu = hello_ppu
IMPORTS = ../spu/hello_spu.a -lspe
include $(TOP)/make.footer
--- end code ---
The reader will notice that the speid in the PPE program will be the same
value as the speid in the main of the SPE.
Also, the arguments passed to the spe_create_thread() are the ones
received by the SPE program when running (argp and envp equals to NULL in
our sample).
Important to remember that when compiled this program will generate a
binary in the spu directory, called hello_spu and another one in the root
directory of this example called hello_ppu, which CONTAINS embedded the
hello_spu.
---[ 2.4.2 - Standard Library Calls from SPE
When the SPE program needs to use any standard library call, like for
example, printf or exit, it has to call back to the PPE main thread.
It uses a simple stop-and-signal assembly instruction with standardized
arguments value (important to remember that since it's needed in
shellcodes for SPE).
That value is returned from the ioctl call and the user thread must react
to that. This means copying the arguments from the SPE Local Storage,
executing the library call and then calling ioctl again.
The instruction according to the manual:
"stop u14 - Stop and signal. Execution is stopped, the current
address is written to the SPU NPC register, the value u14 is
written to the SPU status register, and an interrupt is sent to
the PPU."
This is a disassembly output of the hello_spu program:
---------- begin output ----------
# spu-gdb ./hello_spu
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "--host=powerpc64-unknown-linux-gnu --target=spu"...
(gdb) disassemble main
Dump of assembler code for function main:
0x00000170 <main+0>: ila $3,0x340 <.rodata>
0x00000174 <main+4>: stqd $0,16($1)
0x00000178 <main+8>: nop $127
0x0000017c <main+12>: stqd $1,-32($1)
0x00000180 <main+16>: ai $1,$1,-32
0x00000184 <main+20>: brsl $0,0x1a0 <puts> # 1a0
0x00000188 <main+24>: ai $1,$1,32 # 20
0x0000018c <main+28>: fsmbi $3,0
0x00000190 <main+32>: lqd $0,16($1)
0x00000194 <main+36>: bi $0
0x00000198 <main+40>: stop
0x0000019c <main+44>: stop
End of assembler dump.
(gdb)
---------- end output ----------
---[ 2.4.3 - Communication Mechanisms
The architecture offers three main communications mechanism:
- DMA
Used to move data and instructions between main storage and
a local storage. SPEs rely on asyncronous DMA transfers to hide
memory latency and transfer overhead by moving information in
parallel with SPU computation.
- Mailbox
Used for control communications between a SPE and the
PPE or other devices. Mailboxes holds 32-bit messages. Each
SPE has two mailboxes for sending messages and one mailbox for
receiving messages.
- Signal Notification
Used for control communications from PPE or
other devices. Signal notification (also known as signalling)
uses 32-bit registers that can be configured for
one-sender-to-one-receiver signalling or
many-senders-to-one-receiver signalling.
All three are controlled and implemented by the SPE MFC and it's
importance is related to the way the vulnerable program will receive it's
input.
---[ 2.4.4 - Memory Flow Control (MFC) Commands
This is the main mechanism for the SPE to access the main storage and
maintain syncronization with other processors and devices in the system.
MFC commands can be issued either by the SPE itself, or by the processor
and other devices, as follow:
- A code running on the SPU issue a MFC command by executing a
series of writes and/or reads using channel instructions.
- A code running on the PPU or any other device issue a MFC
command by performing a serie of stores and/or loads to
memory-mapped I/O (MMIO) registers in the MFC.
The MFC commands are then queued in one of those independent queues:
- MFC SPU Command Queue - For channel-initiated commands by the
associated SPU
- MFC Proxy Command Queue - For MMIO-initiated commands by the PPE
or other devices.
---[ 2.4.5 - Direct Memory Access (DMA) Commands
The MFC commands that transfers data are referred as DMA commands. The
transfer direction for DMA commands are based on the SPE point of view:
- Into a SPE (from main storage to the local storage) -> get
- Out of a SPE (from local storage to the main storage) -> put
---[ 2.4.5.1 - Get/Put Commands
DMA get from the main memory to the local storage:
(void) mfc_get (volatile void *ls, uint64_t ea, uint32_t size,
uint32_t tag, uint32_t tid, uint32_t rid)
DMA put into the main memory from the local storage:
(void) mfc_put (volatile void *ls, uint64_t ea, uint32_t size,
uint32_t tag, uint32_t tid, uint32_t rid)
To guarantee the synchronization of the writes to the main memory, there
is the options:
- mfc_putf: the 'f' means fenced, or, that all commands executed
before within the same tag group must finish first, later ones
could be before
- mfc_putb: the 'b' here means barrier, or, that the barrier
command and all commands issued thereafter are NOT executed
until all previously issued commands in the same tag group have
been performed
---[ 2.4.5.2 - Resources
For DMA operations the system uses DMA transfers with variable length
sizes (1, 2, 4, 8 and n*16 bytes (n an integer, of course). There is a
maximum of 16 KB per DMA transfer and 128b aligments offer better
performance.
The DMA queues are defined per SPU, with 16-element queue for
SPU-initiated requests and 8-element queue for PPU-initiated requests.
The SPU-initiated request has always a higher priority.
To differentiate each DMA command, they receive a tag, with a 5-bit
identifier. Same identifier can be applied to multiple commands since
it's used for polling status or waiting on the completion of the DMA
commands.
A great feature provided is the DMA lists, where a single DMA command can
cause execution of a list of transfers requests (in local storage). Lists
implements scatter-gather functions and may contain up to 2K transfer
requests.
---[ 2.4.5.3 - SPE 2 SPE Communication
An address in another SPE local storage is represented as a 32-bit
effective address (global address).
SPE issuing a DMA command needs a pointer to the other SPE's local
storage. The PPE code can obtain effective address of an SPE's local
storage:
--- code ---
#include <libspe.h>
speid_t speid;
void *spe_ls_addr;
spe_ls_addr=spe_get_ls(speid);
--- end code ---
This permits the PPE to give to the SPEs each other local addresses and
control the communications. Vulnerabilities may arise don't matter what
is the communication flow, even without involving the PPE itself.
Follow is a simple DMA demo program between PPE and SPE (see the attached
file for the complete version) - This program will send an address in the
PPE to the SPE through DMA:
--- PPE code ---
information_sent is[1] __attribute__ ((aligned 128)));
spe_git_t gid;
int * pointer=(int *)malloc(128);
gid=spe_create_group(SCHED_OTHER, 0, 1);
if (spe_group_max(gid) < 1 ) {
printf("\nOps, there is no free SPE to run it...\n");
exit(EXIT_FAILURE);
}
is[0].addr = (unsigned int) pointer;
/* Create the SPE thread */
speid=spe_create_thread (gid, &hello_dma, (unsigned long long *) &is[0], NULL, -1, 0);
/* Wait for the SPE to complete */
spe_wait(speids[0], &status[0], 0);
/* Best pratice: Issue a sync before ending - This is good for us ;) */
__asm__ __volatile__ ("sync" : : : "memory");
--- end code ---
--- SPE code ---
information_sent is __attribute__ ((aligned 128)));
int main(unsigned long long speid, unsigned long long argp, unsigned long long envp)
{
/* Where:
is -> Address in local storage to place the data
argp -> Main memory address
sizeof(is) -> Number of bytes to read
31 -> Associated tag to this DMA (from 0 to 31)
0 -> Not useful here (just when using caching)
0 -> Not useful here (just when using caching)
*/
mfc_get(&is, argp, sizeof(is), 31, 0, 0);
mfc_write_tag_mask(1<<31); /* Always 1 left-shifted the value of your tag mask */
/* Issue the DMA and wait until completion */
mfc_read_tag_status_all();
}
--- end code ---
And now between two SPEs (also for the complete code, please refer to the
attached sources):
--- PPE code ---
speid_t speid[2]
speid[0]=spe_create_thread (0, &dma_spe1, NULL, NULL, -1, 0);
speid[1]=spe_create_thread (0, &dma_spe2, NULL, NULL, -1, 0);
for (i=0; i<2; i++) local_store[i]=spe_get_ls(speid[i]); /* Get local storage address */
for (i=0; i<2; i++) spe_kill(speid[i], SIGKILL); /* Send SIGKILL to the SPE
threds */
--- end code ---
--- SPE code ---
/* Write something to the PPE */
spu_write_out_mbox(buffer);
/* Read something from the PPE */
pointer = spu_read_in_mbox();
/* DMA interface */
mfc_get(buffer, pointer, size, tag, 0, 0);
wait_on_mask(1<<tag);
/* DMA something to the second SPE */
mfc_put(buffer, local_store[1], size, tag, 0, 0);
wait_on_mask(1<<tag);
/* Notify the PPE */
spu_write_out_mbox(1);
--- end code ---
------[ 3 - Exploiting Software Vulnerabilities on Cell SPE
I love the architecture manuals and the engineers and the way they talk
about really dumb design choices:
"The SPU Local Store has no memory protection, and memory access wraps
from the end of the Local Store back to the beginning. An SPU program is
free to write anywhere in the Local Store including its own instruction
space. A common problem in SPU programming is the corruption of the SPU
program text when the stack area overflows into the program area. This
problem typically does not become apparent until some later point in the
program execution when the program attempts to execute code in area that
was corrupted, which typically results in illegal instruction exception.
Even with a debugger it can be difficult to track down this type of
problem because the cause and effect can occur far apart in the program
execution. Adding printf's just moves failure point around".
---[ 3.1 - Memory Overflows
In the aforementioned memory design of the SPU is already cleaver that
when an attacker controls the overwrite size it's really easy to exploit a
SPU vulnerability, just replacing the original program .text with the
attacker's one.
It's important to note that the SPU interrupt facility can be configured
to branch to an interrupt handler at address 0 if an external condition is
true (bisled - branch indirect and set link if external data is the
instruction used to check if there is external data available). Since the
memory layout loops around, it's always possible to overwrite this handler
if it's been used.
Another important note is the fact that instructions on memory MUST be
aligned on word boundaries.
There is instruction and data caches for the local storage (depending on
the implementation details), so it's important to assure:
- You are overflowing a large enough amount of data to avoid
caching
- You are not using a self-modifying shellcode unless you issue
the sync instruction (see [13] for references)
---[ 3.1.1 - SPE memory layout
The memory layout for the SPE looks like:
------------------------ -> 0x3FFFF
SPU ABI Reserved Usage
------------------------ | Stack grows from the
Runtime Stack | higher addresses to
------------------------ | the lower addresses.
Global Data |
------------------------ \/
.Text
------------------------ -> 0x00000
For the purpose of test your application, it's really interesting to use the
'size' application:
---------- begin output ----------
# size hello_spu
text data bss dec hex filename
1346 928 32 2306 902 hello_spu
---------- end output ----------
---[ 3.1.2 - SPE assembly basics
It's important in order to develop a shellcode to understand the
differences in the SPE assembly when comparing to PowerPC.
The SPE uses risc-based assembly, which means there is a small set of
instructions and everything in the SPE runs in user-mode (there is no
kernel-mode for the SPE). That said we need to remember there is no
system-calls, but instead there is the PPE calls (stop instructions).
It is also a big endian architecture (keep that in mind while reading the
following sections).
This architecture provides many ways to avoid branches in the code for
maximum efficiency. Since it's not a real problem while exploiting
software, I'll just avoid to talk about and will also avoid to talk about
SIMD instructions. For more informations on that refer to the SPU
Instruction Set Architecture document [12].
---[ 3.1.2.1 - Registers
I already explained a little about the way the architecture works and in
this section I'll just include what is the available register set and how
to use it .
The SPE does not define a conditional register, so the comparison
operations will set results that are either 0 (false) or 1 (true) with the
same width as the operands been tested. This results are used to do
bitwise masking, instruction selection or conditional branching.
As any other platform, there is general purposes registers and special
purpose registers in the SPE:
- General Purpose Registers (0-127) Used in different ways by the
instructions. In the second word of R1 you have the information
about the amount of free space in the stack (the room between
end of the heap and the start of the stack).
- Special Purpose Registers
The SPE also supports 128 special-purpose registers. Some
interesting ones:
* SRR0 - Save and Restore Register 0 - Holds the address
used by the interrupt return (iret) instruction
* LR - Link Register - All branch instructions that set
the link register will force the address of the next
instruction to be loaded on this register
* CTR - Count Register - Usually it's used to hold a loop
counter (like the loop instruction and %ecx register in
intel x86 architecture)
* CR - Condition Register - Used to perform conditional
comparisons
To move data between Special Purpose Registers and General Purpose
Registers we have the instructions
* mtspr (move to special purpose register) mfspr (move from
* special purpose register)
---[ 3.1.2.2 - Local Storage Addressing Mode
In order to address information to/from Local Storage the instructions
uses the following structure:
Instruction_Opcode l10_field RA_field RT_field
8-bit 10-bit 7-bit 7-bit
Where: The signed value of the l10 field is appended with 4 zeros and then
added to the preferred slot in the RA, forcing the 4-rightmost bits of the
sum to zero. After, the 16 bytes of the local storage address are
inserted in the RT field.
Preferred slot for the architecture point of view are the leftmost
4 bytes (not bits).
Important to note here that the IBM convention specifies that:
l10 means a 10-bit immediate value
RA means a general purpose register to be used as
source/destination
RT means a general purpose register to be used as destination
(target)
Knowing that makes it easier to understand why the Local Storage Address
Space is limited to 4 GB.
The actual size of the Local Storage can be viewed accessing the LSLR
(local storage limit register). All effective address are ANDed with the
value in the LSLR before used.
---[ 3.1.2.3 - External Devices
The SPU can send/receive data to/from external devices using the channel
interface. The channel instructions uses quadwords (128bits) to transfer
data to/from general purpose registers and the channel device (which
supports 128 channels).
---[ 3.1.2.4 - Instruction Set
Here are some useful instructions to be used while developing a shellcode
for the SPE.
Instruction Operands Description
Sample
-------------------------------------------------------------------------
lqd (load quadword) rt,symbol(ra) load a value (16 bytes)
from Local Storage (pointed by RA to the general purpose register RT)
lqd $0, 16($1)
stqd (store quadword) rt,symbol(ra) the contents of the
register (RT) are stored at the local storage address pointed by RA
stqd $0, 16($1)
ilh (immediate load halfword) rt,symbol the value of l16 is placed
in register RT
ilh $0, 0x1a0
il (immediate load word) rt, symbol the value of l16 is
expanded to 32bits replicating the leftmost bit and then written to the RT
il $0, 0x1a0
nop (no operation) rt this instruction uses a
false RT and nothing is changed
nop $127
ila (immediate load address) rt, symbol the value of the l18 is
placed in the rightmost 18bits of RT (the remaining bits of RT are zeroed)
ila $3, 0x340
a (add word) rt,ra,rb the operand on register ra
is added to the operand on register rb and the result is written to RT
a $0, $1, $2
ai (add word immediate) rt,ra,value the value (l10 field) is
added to the operand in ra and the result written to RT
ai $1, $1, -32
brsl (branch relative and set link) rt,symbol execution proceeds to the
target instruction and a link register is set (the symbol is a l16 type
and it is extended to the rigth with two 0 bits) - The address of the
current instruction is added to the symbol address for the branch. The
address of the next instruction is written to the preferred byte of the RT
register.
brsl $0, 0x1a0
fsmbi (form select mask for bytes immediate) rt,symbol the symbol is a
l16 value used to create a mask in the register RT copying eight times
each bit. Bits in the operand are related to bytes in the result in a
left-to-right correspondence. fsmbi $3, 0
bi (branch indirect) ra execution proceeds to the
preferred slot of RA. The right two bits in the RA are ignored (supposed
to be zero). There is two flags, D and E to disable and Enable
interrupts.
bi $0
---[ 3.1.3 - Exploiting Software Vulnerabilities in SPE
First of all it's important to make it even more clear that it is
impossible to, for example, force the SPE process to execute a new command
(a.k.a. execve() shellcodes). The same happens for network-based library
functions and others, as already explained we need the PPE to proxy that
for us.
So it open two new paths:
- Create a PPE shellcode to be used while exploiting PPE software
vulnerabilities that will spawn a proxy for commands received by
the SPE and will create a SPE thread to do all the job -> This
is pure PPC shellcode and this article already discussed
everything needed to achieve that. In the attached sources you
have samples in the directory cell-ppe/ [16].
- Create a vulnerability specific code for the SPE, that will
print out internal program information related to the exploited
SPE. This is specially interesting and difficult because:
* Need to remember that the SPE uses instruction-cache, so
sometimes if you overflow just a small amount of bytes,
it will be specially difficult to get it executed
* If you use the wrap-around characteristics of the memory
layout for the SPE, you will probably overwrite also the
information you are interested in.
In the other hand, it's important to say that everything the information
will be in the same place (or easier to understand: there is no ASLR in
the SPE). Running the attached samples (specially the SPE-SPE
communications because it's printing the pointers addresses will make it
clear to the reader).
---[ 3.1.3.1 - Avoiding Null Bytes
It is important to avoid null bytes, so we cannot use the NOP instruction
in our shellcode.
This creates a problem, since the ori instruction will also generate null
byte if used with 0 as an argument (e.g: ori $1, $1, 0).
A good replacement is the instruction or (e.g: or $1, $1, $1) or the usage
of multiple instructions (which will reduce the probability of your return
address).
---[ 3.1.4 - Finding software vulnerabilities on SPE
The simulator provided by IBM has a feature that monitors selected
addresses or regions on the Local Store for read and write accesses. This
feature can help identify stack overflows conditions.o
Invoked from the simulator command windows as follows:
enable_stack_checking [spu_number] [spu_executable_filename]
This procedure uses the nm system utility to determine the area of the
Local Storage that will contain the program code and creates trigger
functions to trap writes by the SPU into this region.
Important to notice that this approach are just looking for writes in the
text and static data and not to the heap. Of course the same approach
used by this feature could be used to help the creation of a fuzzer using
TCL scripts based on the one provided.
------[ 4 - Future and other uses
I can't foresee the future, but this kind of architectures are becoming
more and more common and will open a wide range of new vulnerabilities.
The complexity behind this kind of asymmetric multi-threaded architectures
are even higher than the normal ones. The lack of memory protection will
help also the attackers on how to subvert those systems. The main
processor been based on an already well-known architecture (powerpc) also
helps the dissemination of malicious codes.
Many other researchers are doing stuff using Cell:
- Nick Breese presented on Crackstation project in BlackHat [5]
Basically he used the SIMD capabilities and big registers
provided by the architecture to crack passwords [5]
- IBM Researchers released a study about the usage of the Cell SPU
as a Garbage Collector Co-processor [14]
- Maybe there is JTAG-based interfaces on the cell machines to try
to use RiscWatch [15]
- Unfortunelly the SPU access are controlled by the PPE so run
integrity protection mechanisms from SPU seens infeasible ->
Anyway, I wrote a network traffic analyzer using cell as base
architechture.
------[ 5 - Acknowledgments
A lot of people helped me in the long way for these researches that
resulted in something funny to be published, you all know who you are.
Special thanks to the Phrack Staff for the great review of the article,
giving a lot of important insights about how to better structure it and
giving a real value to it.
I always need to thanks to Filipe Balestra, my research partner, for
sharing with me his ideas, feedbacks, comments and experiences improving a
lot the article and the samples.
I'll never ever forget to say thanks to my research team and friends at
RISE Security (http://www.risesecurity.org) for always keeping me
motivated studying completely new things. Be sure that the unix-asm [16]
project will be updated soon with all the stuff showed here and much more
different types of shellcodes for the architecture. Also, of course the
updates will be available for Metasploit.
Big thanks to the Cell Kernel guru, Andre Detsch for sharing with me his
ideas and discussing the internals of the Linux implementation for Cell.
Conference organizers who invited me to talk about Cell Software
Exploitation, even after many people already talked about Cell they
trusted that my talk was not about brute-forcing (yeah, a lot of fun in
completely different cultures).
To my girlfriend who waited for me (alone, I suppose) during this travels.
It's impossible to not say thanks to COSEINC, for let me keep doing this
research using important company time.
------[ 6 - References
[1] Cell Broadband Engine Architecture, v1.01 October 2006
http://cell.scei.co.jp/pdf/CBE_Architecture_v101.pdf
[2] Sony Computer Entertainment
http://www.sony.com
[3] Toshiba Corporation
http://www.toshiba.com
[4] IBM Corporation
http://www.ibm.com
[5] Breese, Nick; "Crackstation"; Black Hat Europe 2008
http://www.blackhat.com/presentations/bh-europe08/Bresse/Presentation/bh-eu-08-breese.pdf
[6] IBM Power Architecture
http://www-03.ibm.com/chips/power/
[7] IBM Bladecenter QS21
http://www.ibm.com/systems/bladecenter/hardware/servers/qs21/index.html
[8] IBM Roadrunner Supercomputer
http://en.wikipedia.org/wiki/IBM_Roadrunner
[9] The cell project at IBM Research
http://www.research.ibm.com/cell/
[10] Cell Simulator
http://www.alphaworks.ibm.com/tech/cellsystemsim
[11] Cell resource center at developerWorks (SDK download)
http://www-128.ibm.com/developerworks/power/cell/
[12] Synergistic Processor Unit Instruction Set Architecture v1.2
http://www-01.ibm.com/chips/techlib/techlib.nsf/techdocs/76CA6C7304210F3987257060006F2C44/$file/SPU_ISA_v1.2_27Jan2007_pub.pdf
[13] Moore, H.D; "Mac OS X PPC Shellcode Tricks"; Uninformed Magazine 2005
http://www.uninformed.org/?v=1&a=1&t=txt
[14] Cher, Chen-Yong; Gschwind, Michael; "Cell GC: Using the Cell Synergistic Processor as a Garbage Collector Coprocessor"; 2008
http://www.research.ibm.com/cell/papers/2008_vee_cellgc_slides.pdf
[15] RISCWatch Debugger
http://www.ibm.com/chips/techlib/techlib.nsf/products/RISCWatch_Debugger
[16] Carvalho, Ramon de; "Cell PPE Shellcodes"; RISE Security;
http://www.risesecurity.org/papers/lopbuffer.pdf
Others:
PowerPC User Instruction Set Architecture, Book I, v2.02 January 2005
http://moss.csc.ncsu.edu/~mueller/cluster/ps3/SDK3.0/docs/arch/PPC_Vers202_Book1_public.pdf
PowerPC Virtual Environment Architecture, Book II, v2.02 January 2005
http://moss.csc.ncsu.edu/~mueller/cluster/ps3/SDK3.0/docs/arch/PPC_Vers202_Book2_public.pdf
PowerPC Operating Environment Architecture, Book III, v2.02 January 2005
http://moss.csc.ncsu.edu/~mueller/cluster/ps3/SDK3.0/docs/arch/PPC_Vers202_Book3_public.pdf
Cell developer's corner at power.org
http://www.power.org/resources/devcorner/cellcorner/
Linux info at the Barcelona Supercomputing Center website
http://www.bsc.es/projects/deepcomputing/linuxoncell
------[ 7 - Notes on SDK/Simulator Environment
There is some pictures on the simulator and sdk running on the attached file:
images/cell-sim1.jpg and images/cell-sim2.jpg
To install the SDK/Simulator, do:
- Download the Cell SDK ISO image from the IBM alphaWorks website.
- Mount the disk image on the mount directory: mount -o loop
CellSDK<version>.iso /mnt/phrack
- Change directory to /mnt/phrack/software:
- Install the SDK by using the following command and answer any
prompts: ./cellsdk install
To start the simulator: cd /opt/IBM/systemsim-cell/run/cell/linux
../run_gui Click on the 'go' button to start the simulated system
To copy files to the simulated system (inside it run):
callthru source /home/bsdaemon/Phrack/hello_ppu > hello_ppu
Then give the correct permissions and execute:
chmod +x hello_ppu
./hello_ppu
------[ 8 - Sources [cell_samples.tgz]
Attached all the samples used on this article to be compiled in a Linux
running on Cell machine.
Further updates will be available in the RISE Security website at:
http://www.risesecurity.org
For the author's public key:
http://www.kernelhacking.com/rodrigo/docs/public.txt
begin 644 cell_samples.tgz
M'XL(`#EF&$H``^P]:W/;.)+Y:E7-?T`EF5G;D62^*2>9U#E./.-:QW'9SLW-
M;;9<%`G9O%"DAJ3\F*O[[]<`2`E\2K)H64Z`LBV+`!J-;C38W0`:-O:\B\@:
MCCP<[3Q[F"1!,G6=?D+*?]+_95F1#,F`/\HS258467F&]`?")Y/&46R%"#T+
M@R"N*S<K_XDFF^>_@_OCR^9'P?S\UW5#@7*RJDJFX/\J4@G_Z=^+T6C<M9MI
M@S#8T+1*_NN:G/)?-E4B_[JF2,^0U$SS]>D'Y__.=@MMH_U@=.?A08PV][>0
M(DF[Y.'&:>"$[F6`3L=]-[30^]#R[0!MOC_[8.%AX&\A*+7Q-F2E+OHL>]L/
MSD[V/FVC$?8"NVKKAU$V/5M^!R^:VWOM%JM%_#5&SL8O8UBQPVZ5^^X1Y[;
MCT:8/&OAVQB'/H*O%Z,PN`RMX<65Y3L>OH@1&Z71:/RFU7+]&`TMU]^\#EQG
MJ_6_K0WR!!@;CZ,WK0VH[SI0A7Y"<?8`_4H!VR&V8@!X!9\.VI3:Z)<)Z#8Z
M_G)TE/[MR&TDH2T&\.+&<H%<%!)486VUD;Q%X(<X'@/>TIO6_[4>F\$S4HG\
M?[*^X8'KX<;:F"7_BF:F\B_!W`_RKYFJ)N1_%>G#X>D9>DU$8=S:/SC:^XU^
MZURVCCZP;Z_@VQ]>N_/)&K7[D4,EOSNT1JV3T\^_G>Y](B\*4F7RUIAD`$CT
MZU1.6X>?3CZ?GJ>M[4PRNA;J>"!(K70*>+EY_OED:V<(X[`[`++C<-VEZ.FF
M$ODGS&FTC07T/WC_FT3^9=D0^M\J4@7_I\+9@`XX8_Z7)45)^"^;F@'E9$-1
M93'_KR*MG_XW]EUXFGT&#T`MS#ZS^_AB.+!S!4=C\I#!3/5'H@Q>X!B^O<D^
M<JS8RCWR':9/MK+ZY"B$9X/-YU_]7QBDU^CGT5?_>3OY2K3"3!D"FBM#OA;*
M^`Y7PG>R^5$__+8I;:4EDJ_9,D/+\P)[$P1(FQ3DG\T)\4DIK"(UFBKF_T9M
M@%GZOVRF\[]FFII$['](8OY?1:I2UA-;@)D"A^]/]T[_O,##/G;X4EUKHM3_
M"EK[V8=_0E&2L;4#T[4-V4*E7_.4D7_/M;$?X6Y\V^A0GRG_H/,1^=<54]85
MXO_3-$GX_U:2'D/_>^P^BS1-&?FWHN%#K`$N9/\KY/VOF*8L[/]5I`+_P?RZ
MA<^.W%6[4F<4W.!PV3$Q/_]50U4,:O_K@O\K27/QWW-]]E^G[SM18'^S`P<;
M6O=LOC9FO/]U0R/ZORP314`EZW^*0MR`XOV_@K2S_1-YUZ.7A\YK5,EGI$E$
M+>AU9+DCFTA27FOZ:U7Z;Q1:H`B@EP0&@U,-HH..7']\BT[HD-I__W$/'>/X
M)@B_H0B'USA$I"B#0O014"JN8MHJ.J6M@!FQ;X77EG<5H/^T/`^CM[3Y_PC=
M"'0->QRZ\5TW""_?3=$YOW(CP`ETD_`.P;^#{body}amp;,4!8/XQ@KQ&W07C)%M^:"K
M.&X4AVY_'&/DQLCRG9T@9""&@>,.[LC3L>\`EO$51F"_#",4#.B7WXZ_H",<
M02?0;]C'H>6ADW$?=&E6_X@IU<B*T(@\CJ[`B.K?T:H'!)VS!!UT$`+5NP&
M_AN$7<A/,`#:1/`0*5TY;3,!VD9!B#:MF'0D1,&(U-T"[.^09P&.:<UN-4&F
M_7:0ZU/85\$(>G@%4*'/-Z[GH3Y&XP@/QEZ;`8'BZ(_#\]\_?SE'>\=_HC_V
M3D_WCL__?`/%XZL`<O$U9L!<F%E<@`T=!/TPOH,.,!B?/I[N_PZ5]MX?'AV>
M_TDZ<G!X?OSQ[`P=?#Y%>^AD[_3\</_+T=XI.OER>O+Y[&,7H3.,4YHGU"TC
M_(3F`X`Z#("T#HXMUXLX0OP)K(\`5\]!5]8UAB%@8_<:,+5@'([N%F6N%_B7
MM/=0:TKA-\@=(#^(V^@&!B>@'A39S@!->=]&AZ`LMY$N0SG+_P82A<YBJ`%0
M#MP!M'#@!4'81N^#*";%/^TAD$B8/CNR*LD(?3G;2[NY\U/KIU8BXOO!<.2"
MT"1($N)X7G#C`MZ@F0]AR%/JH)?HTK91YP^0,-0)J@2Z6M`S;4\\L\\](ORL
M1O?J.<E#D+J77M#WZ.(]>40^7Y-_,C!?L[*WP$R2?@Y5N3WYP_(\UTORE-WV
MQ<7^WE':P@M
gemini - kennedy.gemi.dev
.$X_6Y;DY*]=D>7Y5?R)JFU5<@V6;;"9T>Q,Y[@T.E!EES(
M(7`K<LQLSI!U"'(TZ(N<QT"%GASO7S#\RY&4^"+PV..+=>F@A,%QJVD#2)*2
M;U8Q"`VGI.J[OE-"*#TAE)%"9R7HP")I&Z85?CS]=G(:$?MO[,5D4OD;AT&$
M;JY@2H!IQ&$CT'/'D`>3S]B.DRF*@8(FM#;\E0WRMT?^2&EF^KF3HD"@)$BJ
M;>EVVL<@9"."9B2YDN8H*4O2:H2-6H99',!,#D<U9<HMCKEZ)=N5RASC/@."
M<*FAX9`R'EY+,?;3[_.,\IKQ?Z].,0P:[I9EVW@T$?WY{body}amp;%U'D;<Y%3<I-<I
MBLYXI)0CJ)CYIW0D%_!.D"*`YJ.+Y793A,PVFY42H/U+3'.D/BD+FHKG9:;@
M%]Q$"P,@F6C)=YB<N^RYWDY^$Y`^IC4X8$FW!EZ80)(*K)#:,.%LDG\*/%#9
M9*0:?'84]U':/,W6U&QV.C@KA52=8]"^*%(=WV+[&I.VY*K1,BDR<Z1TK<AV
M_T;H^0Z(^$[$7I,/J_\O:/_9?IR^E^??&5!O_X'9)^EY^\^0Q?ZOE23F_\V;
M?QDVSS+^`$#.],M6KS'\[,#WL1U3RZ\)PR]%92FS;TFK;VFCKR&;KXH6"UI\
M#1A\S=A[]S'W4AHL:^PU9>L]@*G'NKC3:KUP\,#U052.S\\^[_]S[\.'4QGI
M6MES!>F]_'.RC(N0H;1:]A5,B)P(_^O?OZ(T[0#^<D^%H1SC"%4E0(=\//]Z
M:PZ^@E(.OQ;\WWL^!9(:<T2-2'XK@:C]K[>6]/56D@DP#HCGLO^)T0<O_5I,
M")!=!P!@`*1P0"S'<1F07IL``O5!J@-BID#44B!F`D3:K0*R:WZ]Q7)"%YL#
M$L4WXY0F8'J`+E+='0*D5PN$F*%S`#'J@9@S@9CVUUM%`WKT<RQ.52C0H,(R
MWF0(V^,(6\H=M8(Y.2!R`L0PRX!(#(B69PX'1-,2>@QRF$1V:1\J:6+"D)<K
M:0)&4JC.H`E45I/NR'+I8-.3_NSJE4"@,I:8($I21G:B!!.UK2JZK-1@8BB`
MA4D`$"'D@("5/0%"?HMLYC`!>M@
gemini - kennedy.gemi.dev
!#844JE6&G+BJK5T637F"4[S&RO'_8`
MQ*X'HLP&HG*R@[-`DNZ`["BS94?5ZV0'3,9RX<E*L5+?'7TNFECU0.:;E+1Z
M(,8CS"=:W7P"6F8=D#693Z"R0>2FCB:*.1N(!J/64:N`$)H8]4!XFFA2'4VT
M:L(V0A,5!IO93P'E,+&[Z;N8.#>J@6@P(?4D!@3S([9_B5^1_]]RBM`K6>J]
M*R,LC%@+I%@A2LYN5LFA3I$0YFGR4]<=BHF2=,?A,?&Q]ZJ`B:J48@*5[1XA
M*F!D\"Q.G"VAFG]]EA`6B.K@$G5K*CO`8^:BJ09"!!"S<:+H%0((,#J:F2<+
M-Y\`D'X*)#N?,'</<>=H1F^3>'HJ@6BS9K:JV3
I%Y3JGKM/,3,E@J@9-<(
MH"Y5:TH-"&#JHFJ]$1NZGG!:T/\W\>(N<BYHQOX/@SC[<OX_Q13G?U:2ROU_
M')L7]_[QE4M\?Y-LX?`3#C_A\+N_PX_ZZ2(<DX5VU\EZZLB[7-%FN.ER.A^N
M=;&I\DP76T8/+G7VS:
$R`]4$\4I0H(*$GD9Q80`\P<N=KMV*8_M30AFM:`
M:5I6J0HK4==E1]7REE=3FM8;GLL%)C,N2X_`Y?5CD%9EGU-?JE9MJ#3&H")[
M$G>Y\0@,:I2V<J^.MKV'LO,GM$T5A@QI*1Q=?\PIKA%3?[?,U)]T^5TE{body}amp;'J
MEP#Y#DW]P0Q3G\C@"DU]*I'XUHV+`OF=S';ERT\II6N6GY:>[1[#_KNO_6]H
M3>W_D51=D?/VOZ:*\]\K23/L?\+F)3P`M+KP`0@?@/`!/*0/8"IN^;TZW[V"
M;&CO*H`(!;D(9-#C%.3=#!!GHB#W9NBV!(A1"T2=#>2[59`?^X6^8%IB__><
MIW]GG_^%:2VO_^FF(?2_5:2*\[\9/M_G]&\6P)Q;P,7A7W'X5QS^7<O#OYP\
M5TEY(P=_.8AK=>QWCO.BO<H<L^HDZ9H<^TWFX*=\\M<<2%+-R5])*HX8I>)(
M</[@+Y>CY',F%*5[?4L/"\O\8>$))+VRC>H!9MQG&"6\;>Q@[1(G5XW5GUQM
M\-SJ8YQ:'4^SR\=$Y1GU[_O4JDA-I07MO\$#Q'_2#,DLQ'_25&'_K2)5V'^#
MY>,_Y4&4V(`'KN\D>IJP_X3])^R_];7_!A7!GP8/$/QIT&#PITL<CS`.?6N(
MI]%MJHPY3FOOE:G4J6TX-8>R\*H-/6IQR<7X.OE8.?-$D>(C[TSU<4[5E%,U
MM*C<L3T,S/A5Y:VBQ9?6+59-:DY;KL:J4A4M6"<<>QH-_9,Q85F/E%Y2R[OZ
M>YHWM?7U*37(M$7`MFD1<B9U8A/0FLS(R+6CY?P&/[7D>4PE+?]4Y49V,Z82
M:Z=@*LDBR(\PEY;8_]/4^@\H^X7S/YHB]/^5I`K]/\/G^VC_60!U6X"$TB^4
M?J'TKZ72SPEQE6@WHO`7=9`?6@.90_]H3+F8]_T_Y>;B[YA9^W\-,[__5S)E
M\?Y?2<KM_YVR>8%=O]-*8CNOV,XKMO,N{body}lt;//'<!(':"CP^,O_W5Q\OF/CZ<7
MOT\B]66?3A[3UR.T+_./CD\OR"D=E"0YGT5>*$E6+H\=961YBIK-(^Z'%*2A
M%NK1,ZHDSY1*\B*6"9-\+G/BVX%,2<GT8C_3B\XFZVMGVL&M?.EIQPJE25:N
M/-_97'F6E2O/$R!7GF05H4](4H1.L\IJ)(0JJQ&55.'(EZ\RR=KBJ7J65BJ.
MC3,673SMHI+-2O?JD:1FLUCX;E9+RV:Q@-HL2\]F<>X_9+;*^E4D]03['!UX
MO'/E25:N--^57.DD*U>![V"N`LO*E>=[G2O/LG+E>5+DRG-9A(_8A]=:4QZC
M^^__;>[^'UDS"NN_NKC_9R5I]O[?^WJ`\B#