In my last talk in LibreCon I missed having some step-by-step technical notes, with focus on these issues, to share with the audience and thus help to deploy a minimum sandbox environment in a matter of minutes. Ideally, these notes should also be useful to install common basic tools in a deterministic way.
This entry documents technically how to deploy Ceph RGW/S3 together with an S3 Open Source client (S3cmd) and a command-line packet analyzer (tcpdump) to follow the content of these talks from a practical environment.
The previous week I played a bit with some reliability models and estimators to explore how a drone-based company could run survival analysis over its fleet in order to make decissions on the hardware plane while keeping its operational risks and costs under control.
This entry contains some references, pointers and experiments on applying survival analysis over simulated UAV samples with R
The safe operation of an UAV requires a communication link to handle telemetry data, control commands and other information between the vehicle and the ground control station (GCS).
A simple and affordable way to overcome the range limitation is running the UAVs missions over the mobile radio infrastructure.
This blog post contains technical notes, references and pointers on Open Source UAVs and the feasiblity of these communication links from a vehicle's command and control perspective.
The bug is a stack buffer overflow in the BO pre-processor module included with Snort versions 2.4.0, 2.4.1 and 2.4.2. This vulnerability may be used to completly compromise a Snort sensor.
The entry contains the technical details to get a static Snort vulnerable build together with a new payload disabling the detection capabilities and keeping the NIDS alive after a successful exploitation.
This year’s theme is the application of open source technologies in the industrial and primary sector, as well as the new opportunities that these technologies offer in areas like Cloud Computing, Big Data, Internet of Things (IoT) and the Sharing Economy.
I will be delivering one talk, under the sponsorship of my company Igalia, on Ceph Object Storage and its S3 API. I will introduce the Ceph architecture and the basics to understand how make cloud storage products and services based on Ceph/RGW. I will also comment on the most useful and supported S3 API and tooling working with Ceph.
See you there!
In this post I will explore the model, the API and the snippets of code needed to command, control and monitor a vehicle running Ardupilot with DroneKit-Python.
The event was a nice place to meet some interesting projects and people in the community although, on the low-level technical side, I missed some local company/individual using Open Source flight stacks.
This blog entry contains my notes to build and test an Open Source UAV autopilot based on Ardupilot and Pixhawk, together with the simulation setup needed to explore autonomous missions, flight logs, control commands and open development.
My configuration and scripts to build and run the RISC-V Linux kernel (rev 1.9) on RISC-V QEMU.
As part of this effort and patchset related to the RGW auth subsystem, Radek was kind enough to include my last patches supporting the AWS4 authentication for S3 Post Object API as part of this big patchset.
This entry comments on this AWS4 feature upgrade and how it works with Ceph RGW S3.
This Tuesday, Microsoft released additional updates for older platforms to protect against potential nation-state activity. Among the updates, the official patch fixing the CVE-2017-7269 vulnerability is available.
This official patch is a good way to confirm the root cause of the bug commented on my previous notes, allocating the target buffer size with the number of characters instead of the number of bytes in the functions HrCheckIfHeader and HrGetLockIdForPath.
This blog entry contains my notes running patch diffing on CVE-2017-7269.
This blog post contains my technical notes on the Microsoft Internet Information Services (IIS) 6.0 WebDAV 'ScStoragePathFromUrl' buffer overflow vulnerability (CVE-2017-7269).
While the root cause of this bug is a simple stack overflow, the public proof of concept (PoC) leverages the bug in a creative way to take the control avoiding the different protections and security checks in the operating system.
The result is an interesting sequence of instructions where the original and attack code interleave to run remote arbitrary code.
Some fellows are using the Minio Client (mc) as their primary client-side tool to work with S3 cloud storage and filesystems. As you may know, mc works with the AWS v4 signature API and it provides a modern alternative under the Apache 2.0 License to UNIX commands (ls, cat, cp, diff, etc).
In S3, this feature is used to copy/move data using an existing object as data source in the storage backend instead of downloading/uploading the object to achieve the same effect via the request body.
This extension, part of the Multipart Upload API, reduces the required bandwidth between the RGW cluster and the final user when copying/moving existing objects in specific use cases.
In this post I will introduce the feature to know this concept maps to Ceph and how it works under the hood.
This year I will be attending my first ApacheCon and Apache Big Data here in Europe. Two major events related to open source technologies, techniques and best practices shaping the data ecosystem and cloud computing.
I will be in Seville (Spain) all week, November 14-18, under the sponsorship of my company Igalia. It will be great meeting with some fellows in the Apache Libcloud community, one of the open source projects where I contribute code upstream to support Ceph and custom solutions for cloud providers.
If you are interested in topics such as cloud, distributed systems, data, massive storage, scalability, security, devops, ML... or you would like to chat about some Apache project, feel free to approach me and talk about anything. You can also contact me via mail, linkedin, twitter, etc. See you there!
With AWS Signature Version 4 (AWS4) you have the option of uploading the payload in fixed or variable-size chunks.
This chunked upload option, also known as Transfer payload in multiple chunks or STREAMING-AWS4-HMAC-SHA256-PAYLOAD feature in the Amazon S3 ecosystem, avoids reading the payload twice (or buffer it in memory) to compute the signature in the client side.
Two of the promised promises in Software-Defined Storage (SDS) are higher automation and flexibility in order to reduce the costs around the storage administration tasks.
This entry will go over the required concepts to understand the virtual data and control paths in SDS solutions and how metadata are used to convey the data requirements into the automation software with the proper granularity and flexibility.
This post will introduce the new RGW driver together with the proper configuration parameters to run some examples uploading/downloading objects in Ceph Jewel.
One of the most interesting and powerful features in Ceph is the way how it computes the placement and storage of a growing amount of data at hyperscale.
This computation avoids the need to look up data locations in a central directory in order to allow nodes to be added or removed, moving as few objects as possible while still maintaining balance across new cluster configurations.
Apache Libcloud 1.0.0-rc2 (preview) was released today and it contains the new Outscale storage driver I contributed upstream several days ago.
Along this entry I will introduce the Apache Libcloud project, the Outscale driver and how a new provider can be used to connect with the Outscale object storage service.
In S3, this feature is used to configure buckets in such a way that the user who request the contents will pay transfer fee.
Along this post I will introduce the feature in order to know how this concept maps to Ceph and how it works under the hood.
I will use this blog post to talk about this new feature shipping in Ceph Jewel and the current effort by Outscale and Igalia to raise the level of compatibility between the Ceph RGW S3 and Amazon S3 interfaces.
In detail, I will describe the signing process in AWS4, how it works in Ceph RGW, the current coverage and the next steps in the pipeline around this authentication algorithm.
Over the last few months I have been working in Ceph, a free unified distributed storage system, in order to implement some missing features in RADOS gateway, help some customers with Ceph clusters in production and fixing bugs.
This effort is part of my daily work here in Igalia working in upstream projects. As you could know, Igalia works in the Cloud arena providing services on development, deployment and orchestration around interesting open projects.
My goal with this first post is introducing Ceph in a simple and easy way to understand this marvelous piece of software. I will cover the design and main innovations in Ceph together with its architecture, major use cases and relationship with OpenStack (a well-known free and open-source software platform for cloud computing).
The SDK was designed to support the AWS lifecycle so any possible solution using this SDK will require valid Amazon endpoints or regions to get the things working.
With this post I will have a look in the current Boto3 implementation to know how endpoints and regions are supported in S3, and how it would be possible to use Boto3 with compatible S3 REST interfaces if needed.
I will also comment on setting up new and compatible regions with Boto3 to consume compatible S3 API, and how the current region constraints can be enabled or disabled.
My quick notes and configurations to debug the Windows 10 Kernel on QEMU...