KubeCon / CloudNativeCon Europe 2019

| Comments ()

KubeCon/CloudNativeCon Europe 2019 will be held in Barcelona from May 21 to 23. The event will take place right after Cephalocon Barcelona 2019, which will also be held this year in Barcelona on 19 and 20 May. I will attend both events under the sponsorship of my company Igalia.

The Cloud Native Community Foundation (CNCF) is the open source software foundation organizing KubeCon/CloudNativeCon.

The foundation promotes a cloud native computing vision, universal and sustainable, based on the implementation of applications as microservices, packaged in containers and dynamically orchestrated. All this using an open source stack and looking for resource optimization.

I will follow with special interest the talks related to cloud native storage and the Rook project, an open-source native cloud storage orchestrator for Kubernetes. The Rook operator is compatible with the versions of Ceph Luminous, Mimic and Nautilus, in its first major release.

If you are planning to attend and you want to say hello, do not hesitate to contact me!

Cephalocon Barcelona 2019

| Comments ()

Next week I will attend Cephalocon 2019. It will take place on 19 and 20 May in Barcelona.

I will deliver a talk, under the sponsorship of my company Igalia, about Ceph Object Storage and the RGW/S3 service layer.

In this talk, I will share my experience contributing new features and bugfixes upstream that were developed through open projects in the community.

I will also review some of the contributions from Jewel to Nautilus and its impact from the product/service perspective for users and companies investing in upstream development.

Cephalocon 2019 is our second international conference and it aims to bring together more than 800 technologists and adopters from across the globe to showcase the history and future of Ceph, demonstrate real-world applications and highlight vendor solutions.

The registration of the attendees is still open. You can find more information about the event and how to register on the official event page. The complete schedule is also available.

See you there!

Update 2019/05/25

Ceph Days Galicia 2019

| Comments ()

On Wednesday of last week took place the second Ceph Days Galicia in Santiago de Compostela. It was organized by AMTEGA in collaboration with Red Hat, Supermicro, Colabora Ingenieros, Mellanox, Dinahosting, Aitire and Igalia.

I presented in detail the new archive zone functionality available in Ceph Nautilus. The slides I used in the talk are available here.

If you could not attend and are interested in the topics we talked about, you can read more about the event here. Félix and Camilo have also published a blog post in Spanish about the event.

Thank all the people who participated in the organization and actively collaborated to make the event possible. See you at the next one!

Continue reading ...

RGW/S3 Archive Zone goes upstream in Ceph

| Comments ()

One of my recent contributions, the new Ceph RGW/S3 archive zone, was merged upstream a few days ago and will finally be available in Ceph Nautilus. The feature covers the need to provide archiving zones at the S3 object level in multi-zone RGW configurations.

This blog post describes the feature in detail together with some of the use cases considered during its development.

Continue reading ...

On Ceph RGW/S3 Object Versioning

| Comments ()

In the last few months I spent some time reviewing the Object Versioning feature originally designed for AWS S3, the implementation that is available from Ceph Hammer 0.94 and the user experience with S3 clients.

I found it useful to compile a description of the feature in a short entry, as well as review the main use cases along with examples to have them on hand and share them easily.

The examples use the official AWS CLI (s3api set) and are organized based on the states in which the bucket can be found.

The deployed backend is Ceph RGW S3 / Mimic

Continue reading ...

This entry adds some notes and brief comments about the autopilot interface blocks, the software bus, and the service layer in the context of our Open Source flight platform.

Continue reading ...

Open Source UAV, USS client, driver and controller

| Comments ()

This post contains notes on a common and simplified airspace management interface that interacts with services and features offered by UAV service suppliers (USS).

The entry comments how a USS Web-API can be encapsulated in a "driver", and managed by a USS generic client to be replaced, updated or extended by other "drivers" of the same or different provider easily.

This approach adds new use cases related to the management of USS clients and their economy.

Continue reading ...

This entry reviews public documentation for the technical integration of the flight stack, together with the necessary cooperation and collaboration with third parties, to operate in coordinated and controlled airspaces.

It will have a look in the UTM concept through the NASA-UTM and U-space initiatives, the main blocks that make up the proposed architectures, the UTM services identified and the role of the UAS service provider in these frameworks.

Continue reading ...

Last weekend I did some code tests related to path planning and trayectory control for UAVs. All the tests required working in different coordinate systems and access to geographic information systems.

In detail this post contains my notes on some Python libraries and tools that I found useful related to the WGS84 and UTM coordinate systems, the Digital Elevation Model (DEM), elevation profiles and the Open Source Geographic Information System QGIS.

Continue reading ...

This blog post contains my notes to run an instance of the Software In The Loop (SITL) flight simulator in Docker and how to connect Mission Planner to monitor and interact with vehicle status and data of flight during simulated missions.

The entry also describes how to plan an arbitrary test mission through the flight plan interface offered by Mission Planner.

The MAV Tools, a number of command-line and visual tools for real-time and offline data analysis and plotting, are also used on the data generated by the test mission.

Continue reading ...

Attending Panda Security Summit 2018

| Comments ()

This Friday 18th I will be attending the Panda Security Summit 2018, the first cybersecurity summit organized by Panda Security. This will be my second event of the week in Madrid, the day before I will be attending the AWS Summit Madrid.

The Panda Security Summit (PAAS) will be one-day conference at the Goya Theater. The event will comprise of 6 different talks and 6 workshops on the latest in threats and protection as well as the overall state of cybersecurity.

If you are interested in topics such as kernel programming, assembly, reverse engineering, malware analysis, threat hunting, IDS/IPS, EPP/EDR, ML/DL, Cloud ... feel free to ping me!

Attending AWS Summit Madrid 2018

| Comments ()

This Thursday 17th I will be attending the AWS Summit Madrid. This third edition is celebrated in IFEMA, the Trade Fair Institution of Madrid with more than 35 sessions planned in 7 parallel tracks.

If you are attending the event and would like to chat about AWS, Storage, S3, Data, Security, DevOps, ML/DL, Ceph, etc. Do not hesitate to approach!

Ceph Day in Santiago de Compostela

| Comments ()

This week took place our first Ceph meetup in Santiago de Compostela. The event was organized by AMTEGA in collaboration with Red Hat, Supermicro, Colabora, Cumulus Networks, Dinahosting and Igalia.

My talk Upstream consultancy and Ceph RadosGW/S3 covered the context and value of the upstream contributions in the Ceph project, along with some examples of consulting and technical work that we carried out in Igalia ending with new features and improvements in the project.

Continue reading ...

Ceph RGW/S3 demo container technical notes

| Comments ()

Lately I am attending some industry events and talking about technical aspects related to Ceph RGW, Amazon S3, web APIs, etc.

In my last talk in LibreCon I missed having some step-by-step technical notes, with focus on these issues, to share with the audience and thus help to deploy a minimum sandbox environment in a matter of minutes. Ideally, these notes should also be useful to install common basic tools in a deterministic way.

This entry documents technically how to deploy Ceph RGW/S3 together with an S3 Open Source client (S3cmd) and a command-line packet analyzer (tcpdump) to follow the content of these talks from a practical environment.

Continue reading ...

Open Source UAV and survival analysis with R

| Comments ()

The previous week I played a bit with some reliability models and estimators to explore how a drone-based company could run survival analysis over its fleet in order to make decissions on the hardware plane while keeping its operational risks and costs under control.

This entry contains some references, pointers and experiments on applying survival analysis over simulated UAV samples with R

Continue reading ...

Open Source UAV and mobile cellular networks

| Comments ()

The safe operation of an UAV requires a communication link to handle telemetry data, control commands and other information between the vehicle and the ground control station (GCS).

A simple and affordable way to overcome the range limitation is running the UAVs missions over the mobile radio infrastructure.

This blog post contains technical notes, references and pointers on Open Source UAVs and the feasiblity of these communication links from a vehicle's command and control perspective.

This is the third entry on Open Source UAVs. The previous entries are available here and here.

Continue reading ...

CVE-2005-3252 - Snort 2.4.0-2 remote code execution

| Comments ()

This blog post contains my technical notes on the old Snort Back Orifice (BO) pre-preprocessor buffer overflow vulnerability (CVE-2005-3252).

The bug is a stack buffer overflow in the BO pre-processor module included with Snort versions 2.4.0, 2.4.1 and 2.4.2. This vulnerability may be used to completly compromise a Snort sensor.

This content is part of my interest to design and test more secure networks monitored by Intrusion Detections Systems (IDS) and Intrusion Prevention Systems (IPS) such as Snort and Suricata.

The entry contains the technical details to get a static Snort vulnerable build together with a new payload disabling the detection capabilities and keeping the NIDS alive after a successful exploitation.

Continue reading ...

Attending LibreCon 2017

| Comments ()

This week I will be attending LibreCon 2017, one of the largest international events on open source technologies. It will be held on 19 and 20 October in Santiago de Compostela (Spain).

This year’s theme is the application of open source technologies in the industrial and primary sector, as well as the new opportunities that these technologies offer in areas like Cloud Computing, Big Data, Internet of Things (IoT) and the Sharing Economy.

I will be delivering one talk, under the sponsorship of my company Igalia, on Ceph Object Storage and its S3 API. I will introduce the Ceph architecture and the basics to understand how make cloud storage products and services based on Ceph/RGW. I will also comment on the most useful and supported S3 API and tooling working with Ceph.

See you there!

Open Source UAV API, DroneKit-Python and Geopy

| Comments ()

This blog entry is a continuation of my notes related to Open Source UAV autopilots based on Ardupilot and Pixhawk.

In this post I will explore the model, the API and the snippets of code needed to command, control and monitor a vehicle running Ardupilot with DroneKit-Python.

The entry also contains some technical comments on the communication layer interface (MAVLink), flight use cases implementation and geocoding with Geopy.

Continue reading ...

Open Source UAV Autopilot with Ardupilot and Pixhawk

| Comments ()

Recently I attended a local event related to Drones and Unmanned Aerial Vehicles (UAV) here in Galicia.

The event was a nice place to meet some interesting projects and people in the community although, on the low-level technical side, I missed some local company/individual using Open Source flight stacks.

This blog entry contains my notes to build and test an Open Source UAV autopilot based on Ardupilot and Pixhawk, together with the simulation setup needed to explore autonomous missions, flight logs, control commands and open development.

Continue reading ...

My configuration and scripts to build and run the RISC-V Linux kernel (rev 1.9) on RISC-V QEMU.

Continue reading ...

Some days ago Matt committed the great Radek's effort to have a more coherent and structured scaffolding in the Ceph RGW auth subsystem supporting the differences among the available auth algorithms.

As part of this effort and patchset related to the RGW auth subsystem, Radek was kind enough to include my last patches supporting the AWS4 authentication for S3 Post Object API as part of this big patchset.

This entry comments on this AWS4 feature upgrade and how it works with Ceph RGW S3.

Continue reading ...

CVE-2017-7269 - Binary patch diffing

| Comments ()

This Tuesday, Microsoft released additional updates for older platforms to protect against potential nation-state activity. Among the updates, the official patch fixing the CVE-2017-7269 vulnerability is available.

This official patch is a good way to confirm the root cause of the bug commented on my previous notes, allocating the target buffer size with the number of characters instead of the number of bytes in the functions HrCheckIfHeader and HrGetLockIdForPath.

This blog entry contains my notes running patch diffing on CVE-2017-7269.

Continue reading ...

This blog post contains my technical notes on the Microsoft Internet Information Services (IIS) 6.0 WebDAV 'ScStoragePathFromUrl' buffer overflow vulnerability (CVE-2017-7269).

While the root cause of this bug is a simple stack overflow, the public proof of concept (PoC) leverages the bug in a creative way to take the control avoiding the different protections and security checks in the operating system.

The result is an interesting sequence of instructions where the original and attack code interleave to run remote arbitrary code.

Continue reading ...

Some fellows are using the Minio Client (mc) as their primary client-side tool to work with S3 cloud storage and filesystems. As you may know, mc works with the AWS v4 signature API and it provides a modern alternative under the Apache 2.0 License to UNIX commands (ls, cat, cp, diff, etc).

In the case you are using mc in the client side and Ceph RGW S3 in the server side, you could be experimenting some issues with AWS4 presigned URLs and the error code '403 Forbidden'.

Continue reading ...

The last Upload Part (Copy) patches went upstream in Ceph some days ago. This new feature is available in the master branch now, and it will ship with the first development checkpoint for Kraken.

In S3, this feature is used to copy/move data using an existing object as data source in the storage backend instead of downloading/uploading the object to achieve the same effect via the request body.

This extension, part of the Multipart Upload API, reduces the required bandwidth between the RGW cluster and the final user when copying/moving existing objects in specific use cases.

In this post I will introduce the feature to know this concept maps to Ceph and how it works under the hood.

Continue reading ...

Attending ApacheCon and Apache Big Data Europe 2016

| Comments ()

This year I will be attending my first ApacheCon and Apache Big Data here in Europe. Two major events related to open source technologies, techniques and best practices shaping the data ecosystem and cloud computing.

I will be in Seville (Spain) all week, November 14-18, under the sponsorship of my company Igalia. It will be great meeting with some fellows in the Apache Libcloud community, one of the open source projects where I contribute code upstream to support Ceph and custom solutions for cloud providers.

If you are interested in topics such as cloud, distributed systems, data, massive storage, scalability, security, devops, ML... or you would like to chat about some Apache project, feel free to approach me and talk about anything. You can also contact me via mail, linkedin, twitter, etc. See you there!

AWS4 chunked upload goes upstream in Ceph RGW S3

| Comments ()

With AWS Signature Version 4 (AWS4) you have the option of uploading the payload in fixed or variable-size chunks.

This chunked upload option, also known as Transfer payload in multiple chunks or STREAMING-AWS4-HMAC-SHA256-PAYLOAD feature in the Amazon S3 ecosystem, avoids reading the payload twice (or buffer it in memory) to compute the signature in the client side.

AWS4 chunked upload support is now upstream code in Ceph. It will also ship with the next Jewel release.

Continue reading ...

Two of the promised promises in Software-Defined Storage (SDS) are higher automation and flexibility in order to reduce the costs around the storage administration tasks.

This entry will go over the required concepts to understand the virtual data and control paths in SDS solutions and how metadata are used to convey the data requirements into the automation software with the proper granularity and flexibility.

The entry will also include one of the most simple use cases ('x-amz-website-redirect-location') you can find to illustrate how all these concepts fit in Ceph.

Continue reading ...

The Ansible AWS S3 core module now supports Ceph RGW S3. The patch was upstream today and it will be included in Ansible 2.2

This post will introduce the new RGW S3 support in Ansible together with the required bits to run Ansible playbooks handling S3 use cases in Ceph Jewel.

Continue reading ...

The Ceph RGW storage driver was upstream in Apache Libcloud today. It is available in the Libcloud trunk repository and it will ship with the next release Apache Libcloud 1.0.0.

This post will introduce the new RGW driver together with the proper configuration parameters to run some examples uploading/downloading objects in Ceph Jewel.

Continue reading ...

Scalable placement of replicated data in Ceph

| Comments ()

One of the most interesting and powerful features in Ceph is the way how it computes the placement and storage of a growing amount of data at hyperscale.

This computation avoids the need to look up data locations in a central directory in order to allow nodes to be added or removed, moving as few objects as possible while still maintaining balance across new cluster configurations.

Continue reading ...

Apache Libcloud 1.0.0-rc2 (preview) was released today and it contains the new Outscale storage driver I contributed upstream several days ago.

This release together with the digital signatures are available in the download section. You can read the change log here.

Along this entry I will introduce the Apache Libcloud project, the Outscale driver and how a new provider can be used to connect with the Outscale object storage service.

Continue reading ...

Requester Pays Bucket goes upstream in Ceph

| Comments ()

The last Requester Pays Buckets patches went upstream in Ceph some days ago. This new feature is available in the master branch now, and it will be part of the next Ceph Jewel release.

In S3, this feature is used to configure buckets in such a way that the user who request the contents will pay transfer fee.

Along this post I will introduce the feature in order to know how this concept maps to Ceph and how it works under the hood.

Continue reading ...

AWS Signature Version 4 goes upstream in Ceph

| Comments ()

The first stable AWS4 implementation in Ceph went upstream some days ago. It is now available in the master branch and it will ship with the next Ceph release Jewel as planned.

I will use this blog post to talk about this new feature shipping in Ceph Jewel and the current effort by Outscale and Igalia to raise the level of compatibility between the Ceph RGW S3 and Amazon S3 interfaces.

In detail, I will describe the signing process in AWS4, how it works in Ceph RGW, the current coverage and the next steps in the pipeline around this authentication algorithm.

Continue reading ...

Recent Entries