KubeCon/CloudNativeCon Europe 2019 will be held in Barcelona from May 21 to 23. The event will take place right after Cephalocon Barcelona 2019, which will also be held this year in Barcelona on 19 and 20 May. I will attend both events under the sponsorship of my company Igalia.
The Cloud Native Community Foundation (CNCF) is the open source software foundation organizing KubeCon/CloudNativeCon.
The foundation promotes a cloud native computing vision, universal and sustainable, based on the implementation of applications as microservices, packaged in containers and dynamically orchestrated. All this using an open source stack and looking for resource optimization.
I will follow with special interest the talks related to cloud native storage and the Rook project, an open-source native cloud storage orchestrator for Kubernetes. The Rook operator is compatible with the versions of Ceph Luminous, Mimic and Nautilus, in its first major release.
If you are planning to attend and you want to say hello, do not hesitate to contact me!
In this talk, I will share my experience contributing new features and bugfixes upstream that were developed through open projects in the community.
Cephalocon 2019 is our second international conference and it aims to bring together more than 800 technologists and adopters from across the globe to showcase the history and future of Ceph, demonstrate real-world applications and highlight vendor solutions.
See you there!
On Wednesday of last week took place the second Ceph Days Galicia in Santiago de Compostela. It was organized by AMTEGA in collaboration with Red Hat, Supermicro, Colabora Ingenieros, Mellanox, Dinahosting, Aitire and Igalia.
Thank all the people who participated in the organization and actively collaborated to make the event possible. See you at the next one!
One of my recent contributions, the new Ceph RGW/S3 archive zone, was merged upstream a few days ago and will finally be available in Ceph Nautilus. The feature covers the need to provide archiving zones at the S3 object level in multi-zone RGW configurations.
This blog post describes the feature in detail together with some of the use cases considered during its development.
In the last few months I spent some time reviewing the Object Versioning feature originally designed for AWS S3, the implementation that is available from Ceph Hammer 0.94 and the user experience with S3 clients.
I found it useful to compile a description of the feature in a short entry, as well as review the main use cases along with examples to have them on hand and share them easily.
The examples use the official AWS CLI (s3api set) and are organized based on the states in which the bucket can be found.
This post contains notes on a common and simplified airspace management interface that interacts with services and features offered by UAV service suppliers (USS).
The entry comments how a USS Web-API can be encapsulated in a "driver", and managed by a USS generic client to be replaced, updated or extended by other "drivers" of the same or different provider easily.
This approach adds new use cases related to the management of USS clients and their economy.
This entry reviews public documentation for the technical integration of the flight stack, together with the necessary cooperation and collaboration with third parties, to operate in coordinated and controlled airspaces.
It will have a look in the UTM concept through the NASA-UTM and U-space initiatives, the main blocks that make up the proposed architectures, the UTM services identified and the role of the UAS service provider in these frameworks.
Last weekend I did some code tests related to path planning and trayectory control for UAVs. All the tests required working in different coordinate systems and access to geographic information systems.
In detail this post contains my notes on some Python libraries and tools that I found useful related to the WGS84 and UTM coordinate systems, the Digital Elevation Model (DEM), elevation profiles and the Open Source Geographic Information System QGIS.
This blog post contains my notes to run an instance of the Software In The Loop (SITL) flight simulator in Docker and how to connect Mission Planner to monitor and interact with vehicle status and data of flight during simulated missions.
The entry also describes how to plan an arbitrary test mission through the flight plan interface offered by Mission Planner.
The MAV Tools, a number of command-line and visual tools for real-time and offline data analysis and plotting, are also used on the data generated by the test mission.
This Friday 18th I will be attending the Panda Security Summit 2018, the first cybersecurity summit organized by Panda Security. This will be my second event of the week in Madrid, the day before I will be attending the AWS Summit Madrid.
The Panda Security Summit (PAAS) will be one-day conference at the Goya Theater. The event will comprise of 6 different talks and 6 workshops on the latest in threats and protection as well as the overall state of cybersecurity.
If you are interested in topics such as kernel programming, assembly, reverse engineering, malware analysis, threat hunting, IDS/IPS, EPP/EDR, ML/DL, Cloud ... feel free to ping me!
This Thursday 17th I will be attending the AWS Summit Madrid. This third edition is celebrated in IFEMA, the Trade Fair Institution of Madrid with more than 35 sessions planned in 7 parallel tracks.
If you are attending the event and would like to chat about AWS, Storage, S3, Data, Security, DevOps, ML/DL, Ceph, etc. Do not hesitate to approach!
My talk Upstream consultancy and Ceph RadosGW/S3 covered the context and value of the upstream contributions in the Ceph project, along with some examples of consulting and technical work that we carried out in Igalia ending with new features and improvements in the project.
In my last talk in LibreCon I missed having some step-by-step technical notes, with focus on these issues, to share with the audience and thus help to deploy a minimum sandbox environment in a matter of minutes. Ideally, these notes should also be useful to install common basic tools in a deterministic way.
This entry documents technically how to deploy Ceph RGW/S3 together with an S3 Open Source client (S3cmd) and a command-line packet analyzer (tcpdump) to follow the content of these talks from a practical environment.
The previous week I played a bit with some reliability models and estimators to explore how a drone-based company could run survival analysis over its fleet in order to make decissions on the hardware plane while keeping its operational risks and costs under control.
This entry contains some references, pointers and experiments on applying survival analysis over simulated UAV samples with R
The safe operation of an UAV requires a communication link to handle telemetry data, control commands and other information between the vehicle and the ground control station (GCS).
A simple and affordable way to overcome the range limitation is running the UAVs missions over the mobile radio infrastructure.
This blog post contains technical notes, references and pointers on Open Source UAVs and the feasiblity of these communication links from a vehicle's command and control perspective.
The bug is a stack buffer overflow in the BO pre-processor module included with Snort versions 2.4.0, 2.4.1 and 2.4.2. This vulnerability may be used to completly compromise a Snort sensor.
The entry contains the technical details to get a static Snort vulnerable build together with a new payload disabling the detection capabilities and keeping the NIDS alive after a successful exploitation.
This year’s theme is the application of open source technologies in the industrial and primary sector, as well as the new opportunities that these technologies offer in areas like Cloud Computing, Big Data, Internet of Things (IoT) and the Sharing Economy.
I will be delivering one talk, under the sponsorship of my company Igalia, on Ceph Object Storage and its S3 API. I will introduce the Ceph architecture and the basics to understand how make cloud storage products and services based on Ceph/RGW. I will also comment on the most useful and supported S3 API and tooling working with Ceph.
See you there!
In this post I will explore the model, the API and the snippets of code needed to command, control and monitor a vehicle running Ardupilot with DroneKit-Python.
The event was a nice place to meet some interesting projects and people in the community although, on the low-level technical side, I missed some local company/individual using Open Source flight stacks.
This blog entry contains my notes to build and test an Open Source UAV autopilot based on Ardupilot and Pixhawk, together with the simulation setup needed to explore autonomous missions, flight logs, control commands and open development.
My configuration and scripts to build and run the RISC-V Linux kernel (rev 1.9) on RISC-V QEMU.
As part of this effort and patchset related to the RGW auth subsystem, Radek was kind enough to include my last patches supporting the AWS4 authentication for S3 Post Object API as part of this big patchset.
This entry comments on this AWS4 feature upgrade and how it works with Ceph RGW S3.
This Tuesday, Microsoft released additional updates for older platforms to protect against potential nation-state activity. Among the updates, the official patch fixing the CVE-2017-7269 vulnerability is available.
This official patch is a good way to confirm the root cause of the bug commented on my previous notes, allocating the target buffer size with the number of characters instead of the number of bytes in the functions HrCheckIfHeader and HrGetLockIdForPath.
This blog entry contains my notes running patch diffing on CVE-2017-7269.
This blog post contains my technical notes on the Microsoft Internet Information Services (IIS) 6.0 WebDAV 'ScStoragePathFromUrl' buffer overflow vulnerability (CVE-2017-7269).
While the root cause of this bug is a simple stack overflow, the public proof of concept (PoC) leverages the bug in a creative way to take the control avoiding the different protections and security checks in the operating system.
The result is an interesting sequence of instructions where the original and attack code interleave to run remote arbitrary code.
Some fellows are using the Minio Client (mc) as their primary client-side tool to work with S3 cloud storage and filesystems. As you may know, mc works with the AWS v4 signature API and it provides a modern alternative under the Apache 2.0 License to UNIX commands (ls, cat, cp, diff, etc).
In S3, this feature is used to copy/move data using an existing object as data source in the storage backend instead of downloading/uploading the object to achieve the same effect via the request body.
This extension, part of the Multipart Upload API, reduces the required bandwidth between the RGW cluster and the final user when copying/moving existing objects in specific use cases.
In this post I will introduce the feature to know this concept maps to Ceph and how it works under the hood.
This year I will be attending my first ApacheCon and Apache Big Data here in Europe. Two major events related to open source technologies, techniques and best practices shaping the data ecosystem and cloud computing.
I will be in Seville (Spain) all week, November 14-18, under the sponsorship of my company Igalia. It will be great meeting with some fellows in the Apache Libcloud community, one of the open source projects where I contribute code upstream to support Ceph and custom solutions for cloud providers.
If you are interested in topics such as cloud, distributed systems, data, massive storage, scalability, security, devops, ML... or you would like to chat about some Apache project, feel free to approach me and talk about anything. You can also contact me via mail, linkedin, twitter, etc. See you there!
With AWS Signature Version 4 (AWS4) you have the option of uploading the payload in fixed or variable-size chunks.
This chunked upload option, also known as Transfer payload in multiple chunks or STREAMING-AWS4-HMAC-SHA256-PAYLOAD feature in the Amazon S3 ecosystem, avoids reading the payload twice (or buffer it in memory) to compute the signature in the client side.
Two of the promised promises in Software-Defined Storage (SDS) are higher automation and flexibility in order to reduce the costs around the storage administration tasks.
This entry will go over the required concepts to understand the virtual data and control paths in SDS solutions and how metadata are used to convey the data requirements into the automation software with the proper granularity and flexibility.
This post will introduce the new RGW driver together with the proper configuration parameters to run some examples uploading/downloading objects in Ceph Jewel.
One of the most interesting and powerful features in Ceph is the way how it computes the placement and storage of a growing amount of data at hyperscale.
This computation avoids the need to look up data locations in a central directory in order to allow nodes to be added or removed, moving as few objects as possible while still maintaining balance across new cluster configurations.
Apache Libcloud 1.0.0-rc2 (preview) was released today and it contains the new Outscale storage driver I contributed upstream several days ago.
Along this entry I will introduce the Apache Libcloud project, the Outscale driver and how a new provider can be used to connect with the Outscale object storage service.
In S3, this feature is used to configure buckets in such a way that the user who request the contents will pay transfer fee.
Along this post I will introduce the feature in order to know how this concept maps to Ceph and how it works under the hood.
I will use this blog post to talk about this new feature shipping in Ceph Jewel and the current effort by Outscale and Igalia to raise the level of compatibility between the Ceph RGW S3 and Amazon S3 interfaces.
In detail, I will describe the signing process in AWS4, how it works in Ceph RGW, the current coverage and the next steps in the pipeline around this authentication algorithm.